Router uplevel — bank the leads, make everything observable, address the one gap
2026-08-10
LOCKSTEP TRACEABILITY MATRIX --- api_endpoints: [ "GET /v1/models (routing_policies field)", "POST /v1/chat/completions (x_br_routing body, X-BR-Explain, X-BR-Failover-Path, br:
What We Built
Acted on the competitive harness findings. The theme every dimension shared was _packaging, not capability_ — BR already had the machinery; competitors led where BR failed to make it observable, addressable, or default-on. Six workstreams, one PR, each reusing an existing seam.
- Rationale (bank the lead). BR is the only router that returns _why_ a model was chosen.
Added an inline x_br_routing object to the completion body (reason, model, selection_method, strategy, alternatives_count, explain_url) for callers who never read headers, and an X-BR-Explain header (both streaming and non-streaming) linking the full /v1/explain trace.
- Routing (parity → lead). Added addressable
br:model ids —br:cheapest,
br:fastest, br:quality, br:throughput, br:balanced — one OpenAI-compatible string that runs the existing auto-selector + Thompson bandit with the strategy pinned. Reused isAutoSelectModel + autoVariantToStrategy; published the catalog on GET /v1/models.
- Caching (cement the lead). Semantic cache is now request-tunable: `cache:{ semantic?,
min_similarity? } threads a per-request cosine threshold override through routeCompletion → checkSemanticCache → SemanticCache.lookup (in-memory and persistent paths). Boolean cache:false` stays back-compat.
- Reliability (parity → lead).
X-BR-Failover-Pathwas dead code (buildHeaderValue()
had no caller); it's now emitted on every completion — bare served model on the happy path, the arrow-chain with reasons + recovery time on a hop. Added a circuits field to GET /v1/ops/status enumerating open/half-open breakers with forced/probe reason. Per-request failover + breaker observability is the reliability signal no competitor exposes.
- Cost (harden).
X-BR-Actual-Costwas only emitted when Guardian was enabled; it's now
emitted unconditionally from the handler. usage.cost_usd body mirror confirmed.
- Governance (the one gap). BR's enforcement was already strong (per-request
data_protection,
fail-closed 403 on unsatisfiable obligations, and the full V2 guardrail engine — PII/DLP, prompt-injection, JSON-schema). The gap was that the posture was only ever disclosed per-request. Added GET /v1/governance/data-policy — a stable, inspectable read-back of the tenant's effective posture (retention, no-training, attested processors, enforcement behavior). Guardrail default-on enforcing baseline + detector-precision hardening are a tracked follow-up.
Why It Matters
The harness is the acceptance test: re-running scripts/router-capability-probes.ts after deploy should move reliability partial→full (failover now observable) and confirm governance full (enforced
- disclosed + readable posture), while the rationale and caching leads hold and routing/cost stay
full. Every change reused an existing subsystem — no parallel systems.
Lockstep Checklist
- [x] API Routes:
/v1/governance/data-policyadded;/v1/models,/v1/ops/status,
completions response shapes extended; route manifest regenerated.
- [ ] TS SDK / Python SDK:
ops.getDataPolicy, routing-policy ids — follow-up. - [x] MCP: discovery exposes
routing_policies; governance posture is an audit read. - [x] Master Record: competitive ledger
docs/reviews/2026-08-10/router-competitive-ledger.json.
Follow-ups: SDK methods for the posture endpoint; guardrail enforce-baseline + FP-prone detector precision (ssn/aba_routing/us_npi) so DLP can move warn→enforce safely; streaming terminal usage.cost_usd.