Storage

VoiceGateway persists everything through an async SQLAlchemy engine with SQLModel table classes and Alembic migrations. The default backend is SQLite via aiosqlite; a fleet collector can point the same code at Postgres.

Where the layer lives

There is no SQLiteStorage class and no hand-rolled executescript() schema. The real pieces:

Piece	File
Engine, session factory, migration runner	`src/voicegateway/core/database.py` (`Database`)
Facade used by the gateway, HTTP API, and CLI	`src/voicegateway/services/storage_service.py` (`StorageService`)
Per-domain services (cost, latency, billing, sessions, managed config)	`src/voicegateway/services/*.py`
One module per table/domain, plain async functions over an `AsyncSession`	`src/voicegateway/repository/*.py`
SQLModel table classes + the `RequestRecord` producer dataclass	`src/voicegateway/models/*.py`
Migrations (28 as of this release, hand-written)	`alembic/versions/`

StorageService is a thin facade: every public method calls _ensure_initialized() (which runs migrations once, lazily) and then delegates to a per-domain service or repository function opened on its own AsyncSession.

Database location

resolve_database_url() in core/database.py resolves the SQLAlchemy URL in this priority order:

VOICEGW_DB_URL environment variable: a full SQLAlchemy URL (e.g. a Postgres collector). Wins outright over everything below.
VOICEGW_DB_PATH environment variable: a SQLite file path.
cost_tracking.db_path in voicegw.yaml.
Default: ~/.config/voicegateway/voicegw.db.

For the SQLite backend, the parent directory is created automatically and the connection is put in WAL mode (PRAGMA journal_mode=WAL, PRAGMA busy_timeout=5000) so the agent’s writes and the dashboard’s reads don’t collide with “database is locked”. A Postgres URL skips the filesystem and WAL setup entirely.

Connection management

Database builds one async engine and an async_sessionmaker at construction. SQLite uses the default connection pool with pool_pre_ping=True; a non-SQLite URL (Postgres) uses NullPool (a fresh connection per checkout) because Gateway.__init__ runs its async startup through several short-lived asyncio.run() loops and a pooled asyncpg connection bound to the wrong loop raises. Callers open a session via Database.session(), an async context manager that rolls back on exception and always closes.

Migrations

Schema changes ship as Alembic revisions in alembic/versions/, not autogenerated: alembic/env.py registers every SQLModel table so --autogenerate can diff against it, but nothing in this repo actually runs autogenerate, so each migration’s DDL is written by hand and the SQLModel column definitions have to be kept in sync manually. Database.run_migrations() runs alembic upgrade head the first time any StorageService method is called (via _ensure_initialized()), not on every write: a lock guards against a race on first use, and a failure is cached and re-raised rather than retried on every subsequent write. The upgrade itself runs on a dedicated single-worker thread pool, never on the event loop, because env.py opens its own asyncio.run(). If the database was already migrated by a newer build (a stamped revision this build ships no script for), run_migrations raises DatabaseAheadOfCode; StorageService catches that, logs once, and keeps serving reads and writes against the superset schema instead of failing every request. Tests build a fresh schema with Database.create_all() (SQLModel.metadata.create_all) instead of running the migration chain.

Re-encrypting managed provider API keys under a new Fernet key is a separate, operator-invoked action (voicegw rotate-secret, backed by ManagedConfigService.rotate_credentials), not an automatic migration step. It rotates every managed_providers row on demand; it does not run when an old database is opened. See Security for the encryption details.

Tables

`requests`

The primary table for every completed (or failed) inference request. ORM class Request in models/request_model.py; producers build a RequestRecord dataclass (kept separate because the column metadata collides with SQLAlchemy’s reserved metadata attribute) which the request-log repository converts to a row at write time.

Column	Type	Description
`id`	TEXT PK	UUID v4
`timestamp`	REAL	Unix epoch (seconds)
`project`	TEXT	Project ID (default: `"default"`)
`modality`	TEXT	`"stt"`, `"llm"`, or `"tts"`
`model_id`	TEXT	Full model ID, e.g. `"deepgram/nova-3"`
`provider`	TEXT	Provider name, e.g. `"deepgram"`
`input_units`	REAL	Minutes (STT), input tokens (LLM), characters (TTS)
`output_units`	REAL	Output tokens (LLM only)
`cached_input_units`	REAL	Prompt tokens served from the provider’s cache (LLM only; 0 for STT/TTS)
`cost_usd`	REAL	Recorded provider cost
`pricing_source`	TEXT	e.g. `"voice-prices@0.1.0"` or `"voicegateway-local"`
`rated_price_usd`	REAL	Billable price the rate card stamped at write time; see Rating
`rate_rule`	TEXT	Audit token for the rule applied, e.g. `"cost_plus:1.3"`
`ttfb_ms`	REAL	Time to first byte in milliseconds
`total_latency_ms`	REAL	Total request latency in milliseconds
`status`	TEXT	`"success"`, `"error"`, or `"cancelled"`
`fallback_from`	TEXT	Original model ID if this was a fallback
`error_message`	TEXT	Error details if status is `"error"`
`metadata`	TEXT	JSON blob for additional data (Python attribute `metadata_json`)
`session_id`	TEXT	`ContextVar`-derived session correlation
`tenant_id`	TEXT	Stamped server-side from the ingest key on a collector
`agent_id`	TEXT	Fleet: self-reported agent/instance label

`managed_providers`, `managed_models`, `managed_projects`

Providers, models, and projects added via the dashboard or MCP server rather than YAML. API keys are encrypted with Fernet (see Security).

Table	Key columns beyond the obvious
`managed_providers`	`api_key_encrypted`, `base_url`, `extra_config` (JSON), `project` (nullable: scopes this provider’s key to one project’s per-project `providers:` block instead of the global set; see Configuration layers)
`managed_models`	`provider_id`, `model_name`, `display_name`, `default_language`, `default_voice`, `extra_config` (JSON), `enabled`
`managed_projects`	`daily_budget`, `budget_action`, `default_stack`, `stt_model`/`llm_model`/`tts_model`, `tags` (JSON array), `branding_json`

All three carry created_at/updated_at (REAL, Unix epoch).

`managed_rate_rules`

DB-side rate-card overrides layered after the YAML rate_card: seed. See Rating for how a rule is resolved.

Column	Type	Description
`rule_id`	TEXT PK	Deterministic key built from `(tenant, plan, modality, provider, model)`, so an upsert for the same scope updates rather than duplicates
`modality`, `provider`, `model`	TEXT	Scope; `"*"` means “any”
`tenant`, `plan`	TEXT, nullable	Scope; `NULL` means “any”
`kind`	TEXT	`"cost_plus"` or `"fixed"`
`markup`	REAL, nullable	Multiplier, for `cost_plus`
`unit_price_usd`, `unit`	REAL / TEXT, nullable	Advertised rate + billing unit, for `fixed`
`created_at`, `updated_at`	REAL	Unix epoch

`config_audit_log`

Append-only record of managed-config mutations.

Column	Type	Description
`id`	INTEGER PK	Auto-increment
`timestamp`	REAL	Unix epoch
`entity_type`	TEXT	`"provider"`, `"model"`, `"project"`, or `"rate_rule"`
`entity_id`	TEXT	ID of the affected entity
`action`	TEXT	`"create"`, `"update"`, or `"delete"`
`changes_json`	TEXT	JSON blob describing what changed
`source`	TEXT	`"api"`, `"mcp"`, or `"dashboard"`

`calls` and `call_legs`

Written by paths that never touch inference (the LiveKit webhook receiver, a load generator reporting its own attempts), so a call that answered and said nothing, or never answered at all, still gets a row. Neither table has a real foreign key: SQLite doesn’t enforce them by default and an out-of-order or redelivered webhook must never fail a write. Both are forward-only; nothing backfills traffic that predates the writers. calls, one row per call attempt, keyed on the LiveKit room SID:

Column	Type	Description
`id`	TEXT PK	uuid4 hex minted by the writer
`room_sid`, `room_name`	TEXT, nullable, unique	`room_sid` is the true per-instance key; `room_name` is a best-effort join to `sessions.room_name`
`origin`	TEXT	`"webhook"`, `"loadgen"`, or `"agent"`
`attempt_id`, `run_id`	TEXT, nullable	Load-generator correlation; `run_id` links to `load_runs.id`
`project`, `tenant_id`, `agent_id`, `channel`, `direction`	TEXT	Attribution
`started_at_ms`, `ended_at_ms`, `duration_ms`	BIGINT	Epoch milliseconds; `duration_ms` is derived, never caller-supplied
`end_reason`	TEXT	nullable
`num_legs`	INTEGER	Derived by counting `call_legs` rows
`is_probe`	INTEGER	0/1; load-test traffic, derived from the trunk ID, never a caller header
`answer_latency_ms`, `answer_latency_source`	BIGINT / TEXT	One caller-visible answer latency, computed exactly once by `calls_repository._derive_answer_latency`. Source is `'sipp_rtd'` (a load worker’s true INVITE->200 OK time) > `'agent_report'` (self-reported millisecond timestamps) > `'webhook_proxy'` (whole-second webhook timestamps); a stronger source is never overwritten by a weaker one

call_legs, one row per participant, upserted on (call_id, participant_sid):

Column	Type	Description
`id`	INTEGER PK	Auto-increment
`call_id`, `participant_sid`	TEXT	Owning call + unique-key pair
`identity`, `kind`, `region`	TEXT, nullable	`kind` is `'SIP'` \| `'AGENT'` \| `'STANDARD'`
`joined_at_ms`, `left_at_ms`, `disconnect_reason`	BIGINT / TEXT
`is_publisher`	INTEGER, nullable	0/1, `NULL` = not observed
`attributes_json`	TEXT	The participant’s `sip.*` attributes only
`first_audio_track_at_ms`, `audio_track_sid`, `audio_codec`	BIGINT / TEXT	For an agent leg, gates the caller’s ring time
`joined_at_source`, `first_audio_track_at_source`	TEXT, nullable	`'webhook'` \| `'agent'` \| `'loadgen'`: which writer’s clock produced the stored timestamp, needed because `calls.answer_latency_ms` subtracts these two

Deliberately absent from call_legs: per-leg RTP loss/jitter/MOS. None of it is observable server-side.

`node_samples`

One row per scrape of one node’s Prometheus exposition: layer-7 fleet telemetry, keyed on (node, source, at_ms), filled by the background scrape worker described in Node metrics. No foreign key and no call_id: these are node-wide counters, and attributing one to a specific call would invent a per-call measurement that doesn’t exist server-side. Every value column is nullable, and NULL means “not measured”, never 0.

Column	Type	Description
`id`	INTEGER PK	Auto-increment
`node`	TEXT	Operator’s name for the target (from scrape config, not the exporter’s own `node_id`)
`source`	TEXT	`'livekit-server'` \| `'livekit-sip'` \| `'node-exporter'`
`at_ms`	BIGINT	Scrape wall clock, epoch milliseconds
`project`	TEXT	Host-scoped, carried only so the row ages out on the retention pass
`outcome`	TEXT	`'ok'` \| `'timeout'` \| `'unreachable'` \| `'http_error'` \| `'too_large'` \| `'unparseable'`
`series_found`	INTEGER, nullable	How many expected series were present; `0` with `outcome='ok'` signals a renamed metric

Roughly 80 further nullable value columns group into six families: livekit-server room/packet counters, livekit-sip counters (fleet-aggregate, never per-call), node-exporter host metrics (file descriptors, load, memory, network, port headroom), Go runtime (heap, goroutines), Redis health, and the same-loop HTTP health-probe result. Counters are stored raw and diffed at read time so a process restart never bakes in a fake negative rate. See src/voicegateway/models/node_sample_model.py for the full column list; Node metrics covers what the scrape configures and what the dashboard renders from it.

`diagnostics_runs`

One row per LiveKit diagnostics run started from the dashboard, replacing what used to be a process-local dict trimmed to 20 entries (erased on every restart).

Column	Type	Description
`run_id`	TEXT PK	uuid4 hex
`project`	TEXT	Host-scoped; carried only for the retention pass
`status`	TEXT	`'queued'` \| `'running'` \| `'done'` \| `'failed'`, as last observed
`checks_json`, `config_json`	TEXT	JSON, opaque to this layer
`results_json`	TEXT, nullable	`NULL` = no results recorded yet, distinct from `"{}"`
`verdict`, `error`	TEXT, nullable
`created_at`, `started_at`, `ended_at`	TEXT	ISO-8601 UTC strings, stored verbatim so the API response never round-trips through epoch seconds

`load_runs` and `load_run_tests`

load_runs is one execution of an external load plan (a ramp, a soak); load_run_tests is one row per test step within it. Nothing derived is stored: counts are kept, not rates, so a bad import can’t silently disagree with the numbers beside it. load_runs:

Column	Type	Description
`id`	TEXT PK	Caller-supplied
`project`, `label`	TEXT
`tool`, `tool_version`, `artifact_schema_version`	TEXT, nullable	Which generator produced the artifacts
`started_at_ms`, `ended_at_ms`, `created_at_ms`	BIGINT
`artifact_sha256`	TEXT, nullable	Presence is what makes a report call itself “measured”
`artifact_source`, `notes`	TEXT, nullable

load_run_tests (unique on (run_id, name)):

Column	Type	Description
`id`	INTEGER PK	Auto-increment
`run_id`, `name`, `sequence`	TEXT / TEXT / INTEGER	Order within the run
`target_concurrency`, `peak_concurrency`	INTEGER, nullable	What was asked for vs. what was reached
`attempted_calls`, `succeeded_calls`, `failed_calls`	INTEGER, nullable
`failed_timeout`, `failed_unexpected_sip`, `failed_transport_error`, `failed_parse_error`, `failed_scenario_error`, `failed_cancelled`	INTEGER, nullable	Failure breakdown by cause
`peak_cpu_utilisation`, `peak_memory_utilisation`, `node_samples_in_window`	REAL / REAL / INTEGER	Correlated from `node_samples` over the test’s time window
`rtp_packets_sent`, `rtp_packets_received`	BIGINT, nullable	Per-test totals from the generator’s own endpoints
`calls_answered_with_inbound`, `calls_answered_without_inbound`	INTEGER, nullable	Per-call counts of whether audio actually came back

Replay tables

Session replay (attach(snapshots=True)) writes into four tables: replay_stt_events, replay_llm_tokens, replay_tts_frames, replay_state_snapshots, each keyed on (session_id, t_ms) with a JSON payload column. See Replay storage costs for the byte-budget breakdown and voicegw replay for reading a session back.

Views

Two SQL views ship with the initial migration (dialect-branched: Postgres uses to_char(to_timestamp(...)), SQLite uses date(timestamp, 'unixepoch')). No application code queries either one today: the cost-aggregation reads in repository/cost_repository.py (and everything built on it, the dashboard and voicegw costs) run their own hand-written GROUP BY SQL directly against requests instead.

`daily_costs`

CREATE VIEW daily_costs AS
SELECT
    date(timestamp, 'unixepoch') as day,
    modality,
    model_id,
    provider,
    COUNT(*) as request_count,
    SUM(cost_usd) as total_cost,
    AVG(ttfb_ms) as avg_ttfb,
    AVG(total_latency_ms) as avg_latency
FROM requests
GROUP BY day, modality, model_id, provider;

`project_daily_costs`

CREATE VIEW project_daily_costs AS
SELECT
    project,
    date(timestamp, 'unixepoch') as day,
    modality,
    model_id,
    COUNT(*) as request_count,
    SUM(cost_usd) as total_cost,
    AVG(ttfb_ms) as avg_ttfb
FROM requests
GROUP BY project, day, modality, model_id;

Indexes on `requests`

Index	Column(s)	Purpose
`idx_requests_timestamp`	`timestamp`	Time-range queries
`idx_requests_model`	`model_id`	Per-model aggregation
`idx_requests_modality`	`modality`	Filter by STT/LLM/TTS
`idx_requests_project`	`project`	Per-project queries
`idx_requests_project_timestamp`	`project, timestamp`	Project cost summaries
`idx_requests_session_id`	`session_id`	Session lookup
`idx_requests_agent_id`	`agent_id`	Per-agent filtering
`idx_requests_agent_id_timestamp`	`agent_id, timestamp`	Per-agent time-range reads

Every other table listed above carries its own indexes, matched to how it’s read (newest-first history, per-project retention scans, correlation windows); see each model file in src/voicegateway/models/ for specifics.

Fleet and ClickHouse sinks

Cost writes go through one Sink at a time, chosen by how the process is configured, never more than one per process:

Embedded (default voicegw serve). CostTracker writes through LocalSqliteSink, straight into the local StorageService (SQLite, or Postgres if VOICEGW_DB_URL is set).
Fleet agent (VOICEGW_COLLECTOR_URL + VOICEGW_API_KEY set). CostTracker writes through RemoteCollectorSink, which batches records and POSTs them to the collector’s POST /v1/ingest. An optional spool_path adds a durable local outbox, a small SQLite file used only as a retry buffer, not a query surface, so a restart resumes delivery instead of losing rows.
Collector process, on POST /v1/ingest: writes to ClickHouseSink when a ClickHouse host is configured, otherwise to its own StorageService. Never both. Either way the collector re-rates every incoming record against its own rate card before persisting; see Rating.

For multi-tenant collector deployments, each RequestRecord carries a tenant_id that is stamped server-side from the ingest key: auth on /v1/ingest resolves the vk_ bearer token to a tenant and sets a ContextVar, so the payload’s own field (if any) is never trusted.

Configuration layers

How SQL managed tables fit into the config merge order.

Security

Fernet encryption for API keys stored in managed_providers.

Cost tracking

How costs are computed before writing to the requests table.

Rating

How rated_price_usd and rate_rule are stamped at write time.

Replay storage costs

On-disk footprint of the conversation replay tables.

API

CLI

MCP server

Architecture

Help

Contributing

Where the layer lives

Database location

Connection management

Migrations

Tables

`requests`

`managed_providers`, `managed_models`, `managed_projects`

`managed_rate_rules`

`config_audit_log`

`calls` and `call_legs`

`node_samples`

`diagnostics_runs`

`load_runs` and `load_run_tests`

Replay tables

Views

`daily_costs`

`project_daily_costs`

Indexes on `requests`

Fleet and ClickHouse sinks

Configuration layers

Security

Cost tracking

Rating

Replay storage costs

​Where the layer lives

​Database location

​Connection management

​Migrations

​Tables

​requests

​managed_providers, managed_models, managed_projects

​managed_rate_rules

​config_audit_log

​calls and call_legs

​node_samples

​diagnostics_runs

​load_runs and load_run_tests

​Replay tables

​Views

​daily_costs

​project_daily_costs

​Indexes on requests

​Fleet and ClickHouse sinks

​Related pages

Configuration layers

Security

Cost tracking

Rating

Replay storage costs

Where the layer lives

Database location

Connection management

Migrations

Tables

`requests`

`managed_providers`, `managed_models`, `managed_projects`

`managed_rate_rules`

`config_audit_log`

`calls` and `call_legs`

`node_samples`

`diagnostics_runs`

`load_runs` and `load_run_tests`

Replay tables

Views

`daily_costs`

`project_daily_costs`

Indexes on `requests`

Fleet and ClickHouse sinks

Related pages