costhelm

The LLM gateway that treats cost as a precondition, not a report. Your agents talk to one local endpoint; costhelm owns the keys and refuses the call before the money is gone.

Quickstart View on GitHub

call 4: ollama/phi4:latest — $0.017000 (total $0.047000) — 'Running budget checks before...'
call 5: REFUSED — session:budget-agent-demo would exceed its lifetime budget:
        spent $0.047000 of $0.050000, this call projects $0.016000
        Nothing was sent to a provider; the refusal cost $0.

Govern spend

Hard per-principal budgets over five attribution dimensions — tenant, project, user, agent, session. Every call is projected at worst-case cost and admitted before any provider is contacted; a refusal is an HTTP 402 with the arithmetic in the body, and it costs $0. Budgets →

Route by cost and quality

A small router LLM sizes each prompt; the caller’s declared role clamps the tier with floors and ceilings; candidates are ordered by cost, quality, or a tradeoff dial. Low-confidence answers escalate one tier — scored from structural signals, never a judge LLM. Routing →

See everything

One priced SQLite ledger row per call. OTel spans carrying gen_ai.* attributes plus computed cost. Two zero-build dashboards, and a refusal log for the 402s that by design leave no span. Telemetry →

Eight providers, one API

OpenAI, Gemini (numbered key pool), OpenRouter, Ollama, Groq, Cerebras, NVIDIA, GitHub Models — behind one /v1/chat with capability-aware failover, rate pacing, and failure-class cooldowns, all configured as data. Providers →

The costhelm dashboard: priced ledger, per-principal cost roll-up, budget burn-down and the refusal log

Sixty seconds to a governed gateway

uvx costhelm serve

curl -s localhost:8111/v1/chat -H 'Content-Type: application/json' \
  -d '{"prompt": "hello", "max_tokens": 50}'

Every response carries cost, budget, cache, and router_decision envelopes. Open http://localhost:8111/ for the dashboard, then follow the quickstart to arm a budget and watch it refuse.