Zero cloud dependencies for the LLM: point a native OpenAI-compatible plugin at
a locally running Ollama server instead of a cloud LLM. attach() and
guard() work exactly the same as with any cloud provider.
Prerequisites
Install VoiceGateway for your framework as usual (pip install "voicegateway[livekit]" or [pipecat]). VoiceGateway does not bundle
local-model runtimes; see Installation for the
faster-whisper / kokoro-onnx / piper-tts install commands if you also
want local STT or TTS.
Local LLM: Ollama via the OpenAI plugin
Ollama serves an OpenAI-compatible API, so there is no separate Ollama plugin
to install. Point livekit.plugins.openai.LLM (or
pipecat.services.openai.llm.OpenAILLMService) at it with a custom client:
Wrap it in guard() and pass the session to attach() exactly as with a
cloud provider (see First agent):
attach() identifies a component’s provider from its Python module path, so
an openai.LLM instance is always tagged openai/<model>, even when its
client points at Ollama. Ollama has no billing either way, but the cost row
lands unpriced under openai/qwen2.5:3b rather than as a clean ollama/ $0
row. This is a cosmetic labeling gap, not a cost bug.
Local STT and TTS
VoiceGateway ships no LiveKit- or Pipecat-compatible Whisper or Kokoro plugin.
The whisper / kokoro / piper blocks in voicegw.yaml (see
Providers) back voicegw status and the
dashboard’s connection check only; they do not construct STT or TTS objects
for your session.
To run STT or TTS locally, install the runtime yourself (faster-whisper,
kokoro-onnx, or piper-tts) and wrap it in a class that satisfies your
framework’s STT/TTS interface (livekit.agents.stt.STT / tts.TTS, or
Pipecat’s STTService / TTSService). Tag the model id local/<name> so
attach() prices it at $0 and the dashboard groups it with the other local
rows.
Local STT and TTS trade cost for latency: expect STT and TTS first-byte times
in the seconds, not milliseconds, without a GPU. For most voice agents, cloud
STT/TTS paired with a local LLM is the better cost/latency tradeoff; going
fully local is worth it mainly for air-gapped or privacy-constrained
deployments, where the LLM is usually the largest line item anyway.
- First agent: the full LiveKit agent this pattern drops into.
- guard(): the fallback, rate-limit, and budget signature.
- Providers: every
voicegw.yaml provider block, including the local ones.
