Stochastic Review convergence wave — count truthfulness, cache-hit lineage, savings honesty, data-policy provability, alert deliverability

2026-08-09

routerapiintelligencesecurityinfrasdk

LOCKSTEP TRACEABILITY MATRIX --- api_endpoints: [ "GET /v1/models", "GET /v1/governance/artifacts/{request_id}", "GET /v1/explain/{id}", "GET /v1/intelligence/savings", "GET /v1/ops/alert-log", "GET /v1/ops/status", "POST /v1/chat/completions", ] sdk_methods_updated: [ "ops.getAlertLog / get_alert_log (TS + Py sync+async)", "explain-request data_protection + savings types (TS + Py)", "intelligence savings baseline.methods (TS + Py)", ] mcp_tools_updated: ["none (explain adapter passes fields through untyped)"] ---

What We Built

Five Round-7 blockers, each verified against the live gateway with revert-test discipline.

1. Model-count truthfulness (api-integrator, cost-owner). Docs advertised "44 models" (a count of MODEL_CAPABILITIES) while GET /v1/models served 46 (the PROVIDER_PRICING universe the endpoint registry actually registers). The generator now counts the same source /v1/models serves — 46 models / 9 providers — with a clause that live availability is GET /v1/models (circuit breakers may hide temporarily-unhealthy models). Also corrected the stale agents.json providers_supported list (claimed 5 incl. bedrock/ollama; truth is 9). Deeper capability↔pricing catalog reconciliation (19+15 ID mismatch) is a tracked follow-up with a product decision attached (unservable claude-fable-5/gpt-5.6 capability entries).

2. Cache-hit governance artifact was decision: null (compliance-officer). Exact cache hits returned before the router, so captureLineage never fired and the artifact assembler — which reads only the decision_lineage table — reconstructed a null decision. The cache path now writes a durable decision_lineage row (surface: cache_hit, strategy: cache, zero cost) via the registered lineage-writer seam, and stamps lineageDigest on the audit row so coverage classifies the hit captured instead of sampled_out. (Coverage numbers move — intended parity fix, not a regression.)

3. Savings baseline honesty (cost-owner). x-br-routing-savings baselined against the _costliest considered_ model and, worse, fabricated a global-ceiling baseline when no pool existed. Now it baselines against the ranked runner-up that would actually have served (from bandit scores or threaded eligiblePerfs[1]), labels the method (ranked_runner_up | costliest_considered_declined), returns null rather than fabricate, and discloses the method via X-BR-Routing-Savings-Method + /v1/explain. Recorded savings drop — that is the honesty. /auth/insights/savings savingsVsPremium is relabeled a hypothetical ceiling, not realized savings.

4. data_protection provability (data-protection). Request-level enforcement already existed but was not observable. Added the content-free X-BR-Data-Policy header (both streaming and non-streaming) and a data_protection{enforced,digest,data_policy,summary} block in /v1/explain, so a caller can prove the request-level policy tightened the route without evidence-store access.

5. Alert deliverability (platform-sre) — code half. The SNS alerts topic had zero subscribers in prod (the only subscription was gated on an empty Slack webhook), so alarms fired into the void. Added an unconditional Lambda alert-receipt subscriber (auto-confirms) that persists receipts to Redis, plus GET /v1/ops/alert-log and last_alert_delivery on /v1/ops/status so delivery is provable from outside AWS. Requires an owner-run aws cloudformation deploy of infra/monitoring-stack.yml + a live set-alarm-state proof (Wave 3) — the code and template are ready; the AWS apply is pending.

Why It Matters

Round 7 (valid re-measure) reached mean 8.4 with 5/10 personas at ≥9. These five fixes target the remaining five. Four are code-complete and deploy here; platform-sre closes on the pending CloudFormation deploy. The theme is the review's core standard: not "add a feature" but "make every claim provable on the live system" — honest counts, honest savings, enforceable and observable data policy, cache hits that carry the same evidence as live routes, and alerts that are provably delivered.

Lockstep Checklist

[x] API Routes: /v1/ops/alert-log added; explain/savings/models/governance response

shapes updated; route manifest regenerated.

[x] TS SDK: ops.getAlertLog, explain data_protection/savings, savings baseline.methods.
[x] Python SDK: same, sync + async.
[x] MCP: none needed (explain adapter passes new fields through untyped; alert-log is an ops surface).
[x] Master Record: docs/reviews/2026-08-08/ledger.json (Round 7 recorded; convergence wave tracked).

Follow-ups: (a) owner-run CloudFormation deploy of the alert subscriber + live proof; (b) capability↔pricing catalog reconciliation (A stage 2) with the unservable-model product decision; (c) /auth/insights/savings math rework (labeling shipped, math deferred).