Model catalog was silently truncating responses to 25% of capacity — fixed, with a regression test

2026-08-01

routerprovider-catalog

LOCKSTEP TRACEABILITY MATRIX --- api_endpoints: ["none"] sdk_methods_updated: ["none"] mcp_tools_updated: ["none"] ---

What We Built

MODEL_CAPABILITIES in src/router/provider-catalog-capabilities.ts is a hand-maintained catalog of per-model token limits. It is load-bearing: model-router.ts:659 and :1263 clamp every outbound request's max_tokens to the catalog's maxOutputTokens. The file was last verified 2026-04-15 and had drifted badly.

Claude Opus 4.6 and 4.7 were recorded at 200K context / 32K output when both support 1M / 128K. Sonnet 4.6 was recorded at 16K output against an actual 128K. Haiku 4.5 was at 8192 against 64K. Every request asking for a long response on those models was being silently truncated — no error, no warning, just a shorter answer than the model was capable of and the customer had paid for.

This refresh corrects those limits, adds the twelve current models that were missing entirely (Opus 5, Fable 5, Sonnet 5, Opus 4.8, Opus 4.5, Sonnet 4.5, the GPT-5.6 family), removes seven model IDs that are retired upstream and return 404, and adds a regression test that pins the limits so the next drift fails CI instead of shipping.

Why It Matters

A routing gateway's core promise is that it does not degrade the model you asked for. Capping Opus 4.7 at a quarter of its output budget breaks that promise invisibly — the failure mode is a response that looks complete and isn't. It also made the four newest and most capable Claude models unroutable through the catalog, because they had no capability data at all.

How It Works

The clamp is unchanged; only the data it reads is corrected:

const capInfo = MODEL_CAPABILITIES[endpoint.provider]?.[endpoint.modelId];
if (capInfo && req.maxTokens > capInfo.maxOutputTokens) {
  req.maxTokens = capInfo.maxOutputTokens; // was clamping to a stale ceiling
}

The new test file asserts published limits per model, checks that alias and dated model IDs agree (claude-haiku-4-5 vs claude-haiku-4-5-20251001), asserts retired IDs stay absent, and applies global invariants — positive integer limits, maxOutputTokens <= maxInputTokens — across every provider in the catalog.

One existing test was pinning the bug: model-endpoint-registry.test.ts:552 asserted Opus 4.6 had a 200K context window. Corrected to 1M.

The route-level cap that defeated the catalog

Correcting the catalog alone did not deliver the fix. src/api/routes/completions/index.ts carried a flat ceiling that ran _before_ the router ever saw the request and never consulted the catalog:

if (mt > 32768) {
  body.max_tokens = 32768;
  c.header("X-BR-Param-Capped", "max_tokens=32768");
}

So a caller asking for 128k on Opus 5 was still cut to 32,768 regardless of the catalog. The ceiling is now resolved per model:

const ceiling = resolveMaxOutputTokens(body.model) ?? UNRESOLVED_MAX_TOKENS_CEILING;
if (mt > ceiling) {
  body.max_tokens = ceiling;
  c.header("X-BR-Param-Capped", `max_tokens=${ceiling}`);
}

resolveMaxOutputTokens() accepts provider/model-id or a bare id, and resolves a bare id only when every provider declaring it agrees — ambiguous, unknown, retired and virtual models (auto, aliases) return undefined and fall back to UNRESOLVED_MAX_TOKENS_CEILING (32768). Virtual models are re-clamped against the real endpoint in model-router.ts once routed, so the conservative fallback costs nothing.

Policy note. That flat 32768 originated as a cost-amplification guard (API Security Audit 2026-04-07, #145), so raising it is a deliberate change, not purely a bug fix: worst-case output spend on a single request to a 128k model rises 4×. The judgement is that a model cannot emit more than its own ceiling anyway, and that per-request and per-tenant spend is properly bounded by budget-tracker, budget-tracker-distributed, agent-budget-manager and the community-tier caps in src/api/middleware/community-global-cap.ts — controls actually designed for that job, unlike a constant that silently truncated correct requests. If a hard per-request ceiling is wanted back, it belongs as a tenant-tier policy rather than a global literal.

The Numbers

Model	Before	After
`claude-opus-4-7`	200K in / 32K out	1M / 128K
`claude-opus-4-6`	200K in / 32K out	1M / 128K
`claude-sonnet-4-6`	200K in / 16K out	1M / 128K
`claude-haiku-4-5`	200K in / 8192 out	200K / 64K

Added: claude-opus-5, claude-fable-5, claude-sonnet-5, claude-opus-4-8, claude-opus-4-5, claude-sonnet-4-5 (+ dated forms), gpt-5.6-sol, gpt-5.6-terra, gpt-5.6-luna.

Removed as retired upstream: claude-3-5-sonnet-latest, claude-3-7-sonnet-latest, claude-sonnet-4-0, claude-sonnet-4-20250514, gemini-2.0-flash, gemini-1.5-pro, gemini-1.5-flash.

Verification: pnpm check green, pnpm build green, 94 router test files / 1442 tests passing.

Competitive Edge

OpenRouter and OmniRoute both maintain provider metadata by hand and both drift. The difference here is the regression test: limits are pinned to published values, so drift fails CI rather than silently truncating customer responses. The file header now also records verification status and date per provider, so an unverified entry is visibly unverified rather than implicitly trusted.

Known Gaps

Deliberately not fixed, because authoritative numbers could not be obtained and guessing would recreate the same class of bug:

Gemini 3.x (gemini-3.6-flash, 3.5-flash, 3.5-flash-lite, 3.1-flash-lite, 3.1-pro)

is not in the catalog — Google does not publish per-model token limits on its model pages. Google is now correct but thin: three stable 2.5 entries.

gpt-5.5, gpt-4.1, gpt-4o, o3, o3-mini, o4-mini — could not confirm whether

still served; left in place rather than break working routes.

deepseek, x-ai, groq, perplexity-ai, zai, moonshot (11 models) — unverified, last checked

2026-04-15.

src/security/tool-call-firewall.ts:198-199 downgrades gemini-2.0-pro and

gemini-1.5-pro to gemini-2.0-flash, which is shut down — the firewall's safe fallback points at a dead model. Line 192 also references claude-sonnet-4-5-20241022, which is not a real ID. Both need the Gemini 3.x data above before they can be corrected.

The durable fix is to stop hand-maintaining this file and generate it from each provider's models endpoint. The header now says so.

Lockstep Checklist

[x] API Routes: none changed — this is router-internal data.
[x] TS SDK: not applicable, no API surface change.
[x] Python SDK: not applicable, no API surface change.
[x] MCP Schemas: not applicable, no tool surface change.
[x] Master Record: no new capability; corrects data behind an existing one.