Skip to main content

​
Architecture Overview

VoiceGateway is cost tracking and reconciliation for LiveKit voice agents. It returns native LiveKit STT, LLM, and TTS plugin instances across cloud providers and local models, with modality-aware unit accounting (audio-minutes, tokens, characters), resolver-time fallback chains, rate limiting, budget enforcement, a voicegw reconcile command, and a web dashboard.

​
System Architecture

​
Request Flow

Every call to gateway.stt(), gateway.llm(), or gateway.tts() follows the same path:

​
Directory Structure

The Python package lives under voicegateway/. A separate dashboard/ tree carries the FastAPI backend and the React + TypeScript + Vite + Recharts frontend for the local cost dashboard. 
voicegateway/
├── core/
│   ├── gateway.py
│   ├── config.py
│   ├── config_manager.py
│   ├── registry.py
│   ├── schema.py
│   └── crypto.py
├── inference/
│   ├── __init__.py
│   ├── _factory.py
│   ├── _project.py
│   ├── _session_context.py
│   ├── _resolution.py
│   ├── _stt.py
│   ├── _llm.py
│   └── _tts.py
├── providers/
│   ├── base.py
│   ├── openai_provider.py
│   ├── deepgram_provider.py
│   ├── cartesia_provider.py
│   ├── anthropic_provider.py
│   ├── groq_provider.py
│   ├── elevenlabs_provider.py
│   ├── assemblyai_provider.py
│   ├── ollama_provider.py
│   ├── whisper_provider.py
│   ├── kokoro_provider.py
│   └── piper_provider.py
├── middleware/
│   ├── cost_tracker.py
│   ├── latency_monitor.py
│   ├── rate_limiter.py
│   ├── logger.py
│   ├── budget_enforcer.py
│   └── instrumented_provider.py
├── storage/
│   ├── sqlite.py
│   └── models.py
├── server.py
├── mcp/
│   ├── server.py
│   ├── auth.py
│   ├── errors.py
│   ├── schemas.py
│   └── tools/
└── pricing/
    └── catalog.py
dashboard/
├── api/
└── frontend/

​
Design Principles

  1. Async throughout — all database, HTTP, and provider operations use async/await. The Gateway provides synchronous wrapper methods for convenience.
  2. Lazy loading — providers are only imported and instantiated on first use. pip install voicegateway[openai] installs only the OpenAI SDK.
  3. Transparent instrumentation — InstrumentedSTT/LLM/TTS wrappers proxy all attribute access via __getattr__, so user code sees the exact same API as the underlying provider instance.
  4. Config layering — three sources merged at startup: environment variables (highest priority), SQLite managed tables (dashboard/MCP writes), and YAML (base config). Each resource carries a source field ("yaml" or "db").
  5. Encryption at rest — all API keys stored in SQLite are encrypted with Fernet (AES-128-CBC + HMAC-SHA256). Keys in API responses are masked to secr...2345 format.

​
Key Components

ComponentFilePurpose
Gateway Corecore/gateway.pyMain orchestrator, entry point for all requests
Provider Abstractionproviders/base.pyABC for all 11 provider implementations
Middlewaremiddleware/Cost, latency, rate limiting, fallback, budget
Cost Trackingpricing/, middleware/cost_tracker.pyPer-request cost calculation, pricing layer, streaming validation
Storagestorage/sqlite.pySQLite schema, tables, views, indexes
Config Layerscore/config_manager.pyYAML + SQLite + env merge strategy
Securitycore/crypto.pyFernet encryption, secret management, masking