Skip to content
costhelm

Configuration

How config resolves

Section titled “How config resolves”

Every config file resolves the same way, most specific first:

  1. Env-var path override — COSTHELM_ROUTING_YAML=/path/to/routing.yaml
  2. User config dir — ~/.costhelm/routing.yaml (override the dir with COSTHELM_CONFIG_DIR)
  3. Packaged default — shipped inside the wheel

Check what won:

Terminal window
costhelm paths
config dir : /Users/you/.costhelm  (override: COSTHELM_CONFIG_DIR)
pricing.yaml : .../costhelm/economics/pricing.yaml  [packaged default]
budgets.yaml : /Users/you/.costhelm/budgets.yaml    [user config dir]
routing.yaml : .../costhelm/routing/routing.yaml    [packaged default]
cache.yaml   : .../costhelm/cache/cache.yaml        [packaged default]

Malformed config raises rather than silently degrading — a typo in a budget ceiling must not turn into “no budget”. A broken file is reported on the console and surfaced in app.state.config_errors (and on /v1/budget for budgets), while the rest of the gateway keeps serving.

The four files

Section titled “The four files”
File Governs Ships as
pricing.yaml $/Mtok per model, glob patterns, cache/batch multipliers, quality weights full table
budgets.yaml Spend ceilings per principal empty — nothing refused
routing.yaml Tiers, roles, ladders, escalation, provider limits:, agent_pins: full policy
cache.yaml Semantic response cache disabled

Full annotated copies (synced from the shipped defaults) are in the configuration reference.

Secrets and switches (.env)

Section titled “Secrets and switches (.env)”

Provider keys and runtime switches live in the environment (a repo-local .env is loaded at boot). A provider registers only when its key variable is set — a misspelled variable fails silently, so after editing, check curl -s localhost:8111/v1/providers.

.env.example
# Copy to .env and replace only the providers you use. Never commit .env.
#
# A provider registers ONLY if its key variable is set (costhelm/providers.py).
# A missing or misspelled name fails SILENTLY: the gateway starts fine and just
# omits that provider, so the first symptom is `unknown provider 'x'` on a call
# that should have worked. After editing, restart the gateway and check
#   curl -s localhost:8111/v1/providers
# lists what you expect — load_dotenv() runs at import, so a gateway started
# before you edited .env keeps serving the old environment forever.


GEMINI_API_KEY_1=replace-me
GEMINI_API_KEY_2=replace-me
# ...up to GEMINI_API_KEY_16 (MAX_GEMINI_KEYS). Numbered slots become a pool of
# gemini_1..gemini_N. A single bare GEMINI_API_KEY is accepted as a 1-member pool.


# --- Other providers. Omit any you do not have a key for. -------------------
# The name is NOT what you would guess, and fails silently if guessed:
#   OPEN_ROUTER_API_KEY  — underscore between OPEN and ROUTER, though the
#                          provider is called `openrouter` everywhere else.
OPEN_ROUTER_API_KEY=replace-me
# OpenAI itself, not one of the servers that merely speak its dialect. This
# provider is never free at any tier and its models bill frontier rates — set
# it deliberately, not by reflex.
OPENAI_API_KEY=replace-me


# --- Providers out of the default order --------------------------------------
# Supported by code but not in the default routing order. Set a key AND add the
# name to COSTHELM_ORDER to enable one. Note GITHUB_ACCESS_TOKEN is a GitHub
# PAT with Models access (NOT GITHUB_API_KEY).
# GROQ_API_KEY=replace-me
# CEREBRAS_API_KEY=replace-me
# NVIDIA_API_KEY=replace-me
# GITHUB_ACCESS_TOKEN=replace-me


# Logical provider order. Numbered Gemini slots are expanded automatically.
# Entries whose provider did not register are skipped, so listing a provider
# here does NOT make it available — only its key does. (Ollama additionally
# needs OLLAMA_MODEL below, or it never registers at all.)
COSTHELM_ORDER=ollama,gemini,openrouter,openai
COSTHELM_PORT=8111


# --- Default model per provider (all OPTIONAL) ------------------------------
# These set the model used when a request does not pin one. A caller that sends
# an explicit "model" in the /v1/chat body always wins, so a client with its
# own tier ladder does not need any of these set.
#
# GEMINI_MODEL=gemini-2.5-flash
# OPENROUTER_MODEL=meta-llama/llama-3.3-70b-instruct
# GROQ_MODEL=openai/gpt-oss-120b
# CEREBRAS_MODEL=zai-glm-4.7
# GITHUB_MODEL=openai/gpt-4.1-mini
# NVIDIA_MODEL=meta/llama-3.1-8b-instruct
# OPENAI_MODEL=gpt-5.6-terra
#
# Ollama registers only when OLLAMA_MODEL is set — there is no default.
# OLLAMA_MODEL=gemma4:31b
# OLLAMA_URL=http://localhost:11434


# --- Ledger -----------------------------------------------------------------
# The per-call ledger lives at ~/.costhelm/gateway.sqlite by default.
# COSTHELM_GATEWAY_DB=$HOME/.costhelm/gateway.sqlite


# --- OpenTelemetry -> Jaeger (deploy/docker-compose.observability.yml) -------
# OTel *base* URL — costhelm appends /v1/traces itself. Use 4318 (OTLP/HTTP);
# gRPC (4317) would need opentelemetry-exporter-otlp-proto-grpc, not a
# declared dep.
# OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
# OTEL_SERVICE_NAME=costhelm-gateway
# COSTHELM_TRACE_UI=http://localhost:16686
#
# Prompt/completion text on spans: the highest-PII payload the gateway handles,
# so the shipped default stays OFF. Opt in only against a localhost collector
# holding test data; never point it at a shared collector.
# COSTHELM_OTEL_CAPTURE_CONTENT=1