Home Dashboard GitHub

Self-reporting honesty pass: advertised catalog matches the live endpoint, header matches body, timestamps agree

2026-08-11

routerapisecuritydiscovery

LOCKSTEP TRACEABILITY MATRIX --- api_endpoints: ["GET /v1/discovery", "GET /deploy.json", "POST /v1/chat/completions", "GET /attestation"] sdk_methods_updated: ["none"] mcp_tools_updated: ["none"] ---

What We Built

A remediation pass driven by a live stochastic review (10 adversarial personas scoring the running gateway, not mocks). Each finding was verified against the deployed build before and after the fix. Five classes of self-reporting dishonesty were closed:

  1. Advertised catalog now equals what GET /v1/models serves (45, not 46). The pricing

catalog carried three gemini rows (gemini-3.1-pro, gemini-3-flash, gemini-2.0-flash) that were priced but had no MODEL_CAPABILITIES entry, so they were never registered as endpoints — inflating the marketed count. They're removed. The count now _includes_ the two embedding models because /v1/models returns them, so an agent counting the endpoint and an agent reading the docs get the same number. Reconciled across README, llms.txt, llms-full.txt, ai-plugin.json, the homepage, both SDK READMEs, and the drift gate.

  1. GET /v1/discovery by_provider is internally consistent. It was keyed on the tenant's

configured provider keys (4) while total counted the whole catalog (45); the per-provider breakdown now derives from the served catalog, so providers = 9 and by_provider sums to total.

  1. X-BR-Routed-Model and the response body model agree. DeepSeek's deepseek-reasoner

is an alias of deepseek-v4-flash; the provider echoed the canonical id in the body while the header reported the routed alias. The body is now normalized to the routed model on both the streaming and non-streaming paths, and the provider's canonical id is preserved on a new X-BR-Upstream-Model header for auditability.

  1. X-BR-Complexity-Level matches the difficulty routing acts on. A terse-but-hard prompt

(a one-line proof request) scored "moderate" because structural signals dominated the content signal; a strong formal-reasoning content score now floors the level at "complex".

  1. **/attestation, /deploy.json, and /v1/ops/build agree on build identity and never go

null in prod.** They read one resolver now; deploy.json gains a real previous_commit_sha / deployed_at from deploy-time env, so "did the last deploy change anything?" is answerable.

Why It Matters

An autonomous agent decides whether to trust a platform by cross-checking its own claims. A catalog number that says 46 while the endpoint serves 45, a routing header that names a different model than the body, or a green attestation that can't say what changed — each is a small lie that a careful machine consumer catches and penalizes. This pass makes every load-bearing self-description verifiable against the running system.

How It Works

scripts/catalog-counts.ts is the single source of truth for the advertised count and now counts every registered pricing row (the 2026-08-01 capability refresh guarantees each priced row is registered). The completions path resolves alias→canonical _once_ and reports one identifier to the caller while surfacing the upstream id separately. src/infra/build-info.ts gains a committed-constant fallback (BUILD_INFO.built_at) so the timestamp is non-null in the bundled prod image, and /v1/ops/build reads the same resolver.

Competitive Edge

Portkey and OpenRouter advertise catalogs and routing decisions but don't expose a per-request, cross-checkable decision trace whose numbers reconcile with the discovery endpoint, the response headers, and the attestation surface. Provable self-reporting is the governance-plane differentiator.

Lockstep Checklist

  • [x] API Routes: src/api/routes/completions/* (header agreement) and

src/api/capabilities/system/discovery.ts updated. Response _shapes_ unchanged; only values reconciled + one additive response header (X-BR-Upstream-Model).

  • [x] TS SDK: No change required — no new methods or request/response body fields; the new

X-BR-Upstream-Model is an additive response header the SDK passes through.

  • [x] Python SDK: No change required (same rationale); README model count regenerated to 45.
  • [x] MCP Schemas: No change required — no tool signatures changed; br_list_models

already returns the live catalog (45).

  • [x] Master Record: No new capability; existing routing/discovery/attestation capabilities

unchanged in shape.

Round 2 — the second review wave

After the first batch deployed (7b9fc6c), the full 10-persona review reran live. Six personas hit 9; four remained below and drove a second remediation batch, each fix verified against the running build's exact finding:

  1. cost-owner (8→): honest aggregate savings baseline. GET /v1/intelligence/savings labeled

baseline.model = "openai/gpt-5.5" (the global costliest) and stamped every byPath[].from with it, contradicting the per-request ranked_runner_up baseline shown on /v1/explain and the response header. The dollars were always the honest recorded per-request values — only the label lied. Now byPath[].from is the real recorded runner-up, baseline.model is null for the mix, the global ceiling moved to a clearly-separate reference_ceiling_model, and the ranked_runner_up vs costliest_considered_declined split is surfaced. Totals unchanged.

  1. data-protection (6→): denied requests and PII are now provable after the fact. A

data-protection-denied request (403) wrote no record, so /v1/explain 404'd — the enforced-deny event existed only in transient headers. Denials now persist a content-free record (enforced provider-filter, data_policy lattice value, digest, deny reason). And outbound PII/guardrail actions are now recorded into the decision trace and surfaced as guardrail_actions[] + a denial field on /v1/explain (non-streaming; streaming PII persistence is a noted follow-up).

  1. dashboard-user (5→): /auth/provision CORS on error responses. The 401 lacked

Access-Control-Allow-Origin (preflight had it), so the browser blocked the body and onboarding showed a generic failure. ACAO now applies to all /auth/ responses except the genuinely credentialed paths (/auth/events, /auth/admin, /auth/scim), which keep echoed-origin + credentials. (A separate frontend onboarding gap — /auth/provision requires a Supabase token, so a logged-out user needs a login screen — is a site/dashboard change on the Vercel deploy target, tracked separately.)

  1. selection-method vocabulary honesty. /v1/discovery advertised thompson-sampling as the

method while the live X-BR-Selection-Method emits capability-matched — a value absent from the documented enum. The discovery contract now leads with capability-matched (the actual always-on method: required-tier floor + shrinkage-blended learned reward + ε-greedy exploration).

Still open after this batch: compliance-officer (boundRatio=0 / policy_context: null — no tenant traffic bound to a framework pack; a governance-binding follow-up), the data-protection processor-facts deployment config (no attested zero-retention endpoints on the deployment, so constrained requests can't route), and the dashboard frontend onboarding screen.