AI Assistant — Voice-Enabled Task Orchestration Engine

The Challenge

Executing multi-step automated desktop tasks via voice without unconstrained execution risks — ensuring deterministic tool boundaries and graceful error recovery.

The Solution

Built a Python task orchestration engine leveraging OpenAI Function Calling with strict JSON Schema validation and Pydantic model enforcement for every tool invocation, with fallback exception handlers to prevent cascading failures.

Tech Stack & AI Pipeline

Python handles audio capture via SpeechRecognition, transcription via Whisper, and tool dispatch via OpenAI's function-calling layer with strict schema enforcement. Each tool is registered with a JSON Schema contract, ensuring the LLM can only invoke validated actions.

Orchestration Architecture

      graph TD
          Voice[Voice Input / Microphone] -->|1. Audio Capture| Whisper[Whisper Transcription]
          Whisper -->|2. Text Intent| LLM[OpenAI Function Calling Engine]
          LLM -->|3. JSON Schema Validated Call| Router{Tool Router}
          Router -->|Email Tool| Email[Send Email Action]
          Router -->|File Tool| File[File System Action]
          Router -->|Search Tool| Search[Web Search Action]
          Router -->|Fallback| Error[Graceful Error Handler]