Routing & tiers

Routing answers two different questions, on two different ladders:

• How big is this prompt? — the size ladder: TINY → LARGE → HUGE
• How good must this answer be? — the capability ladder: CHEAP → STANDARD → FRONTIER, ordered by blended $/Mtok

Classification

A separate pool of small, fast “router LLMs” — with its own quotas and its own ledger rows — receives a bounded envelope (token count + a head/tail sample, never the full prompt) and emits one word: TINY, LARGE, or HUGE. If no router is available, a token-count rule decides. Prompts estimated above 8k tokens skip the LLM entirely.

Roles clamp the tier

The caller declares a role via auto_route; the role — not the prompt — gets the final say:

roles:
  perception:                 # cheap, high-volume observation
    default_tier: TINY
    max_tier: LARGE           # ceiling: cannot reach the frontier
    escalate: true
  decision:                   # the step whose mistakes are expensive
    min_tier: LARGE           # floor: a TINY classification is overridden
    default_tier: LARGE
  adjudicator:                # frontier, always
    min_tier: FRONTIER
    default_tier: FRONTIER
    escalate: false

A role’s default_tier selects its ladder, so worker (default_tier: STANDARD) routes across models while perception routes across prompt sizes — one gateway holds both ladders without either seeing the other’s rungs.

Candidate ordering

Each tier declares a provider ring; the ring can be re-sorted using the prices and quality weights already in pricing.yaml:

selection:
  objective: cost_quality   # order | cost | quality | cost_quality
  tradeoff: 0.5             # 0.0 = pure cost, 1.0 = pure quality

Callers can override the dial per call (cost_quality_tradeoff), and a tier can pin objective: order — a one-model rung must not be re-sorted into a different model. strict: true keeps a rung a rung: without it, other configured providers are appended as a last-resort tail.

HUGE filters candidates by real context windows (min_ctx vs each provider’s max_ctx), so an over-long prompt gets a refusal naming the limit, not a mystery 503.

Everything is data

Tiers, roles, ladders, benched providers, cooldowns, provider limits, agent pins — one file, routing.yaml. Adding a role is an edit, not a deploy; no Python knows any role’s name.