Home Dashboard GitHub

Closed the cache bypass of provider.only — caches now carry processor provenance

2026-08-08

routersemantic-cacheprompt-cachedata-protection

LOCKSTEP TRACEABILITY MATRIX --- api_endpoints: ["none"] sdk_methods_updated: ["none"] mcp_tools_updated: ["none"] ---

What We Built

provider.only / provider.ignore became an enforced routing constraint in 33b4b21ee: selection filters to the named processors and the B3 endpoint boundary refuses whatever the caller excluded. Live verification on c66491f found the constraint was still bypassable — not at the router, but around it.

A prompt sent once WITHOUT a filter and then again WITH provider.only: ["anthropic"] returned HTTP 200, x-br-cache: hit, x-br-cache-source: store, body from gpt-4o-mini-2024-07-18. The cache answers before the router is ever called, so a caller who restricted routing to an approved sub-processor received content produced by an excluded one, with no warning. For a data-protection control that is the whole ballgame.

Both caches are now filter-aware, and both do it by PROVENANCE: every entry records the processor that produced it, and a hit is served only when that processor is permitted by the CURRENT request's filter.

Why It Matters

A sub-processor allowlist that a cache can reach around is not an allowlist. The customer-visible promise — "content from this request only goes to processors I named" — was true of the network call and false of the response body. Buyers running DPIAs test exactly this: they send the prompt twice.

The fix also refuses to trade the control for cache performance. Filtered requests keep hitting cache whenever the producing processor is one they approved, which is the common case (a caller pinning only: ["anthropic"] mostly talks to Anthropic).

How It Works

Provenance, not keying. Folding the filter into the cache key would also be safe, but it partitions the cache per filter: a request with only: ["openai"] would take a permanent fresh miss against an entry OpenAI itself produced. Recording the producer turns the decision into a fact about the entry instead of a guess encoded in a hash — and it is what makes the refusal explainable.

Three rules, shared by every read path via src/router/provider-filter.ts:

  1. No active filter ⇒ hit as before. Caching is untouched for most traffic.
  2. Active filter + known producer ⇒ ordinary allow/deny membership test.
  3. Active filter + UNKNOWN producer ⇒ refused. Absence of provenance is not

permission — the same rule that separates not_configured from ok in the health check and not_scanned from clean in the guardrail summary.

Where provenance lives:

  • Semantic cache (model-semantic-cache.ts): CacheEntry.provider, written

by the router; ${provider}/${modelId} on the entry's model identity is the fallback for older entries. The pgvector tier is checked the same way from the row's model identity. policyDigest was deliberately NOT reused — it answers "same policy posture?", which is a different (and much blunter) question than "who produced this?", and folding the filter into it would have made every filtered request miss.

  • Exact prompt cache (api/middleware/prompt-cache.ts): the in-memory LRU

entry carries provider, the durable row gets prompt_cache.provider (schema v66). Rows written before v66 are NULL ⇒ unknown ⇒ refused while a filter is active, then re-earned with real provenance on the next miss. The gap self-heals; it is not backfilled, because a guessed producer is exactly the false provenance the column exists to prevent.

  • Router cache layer (model-cache-layer.ts, not currently wired into the

gateway) got the same guard so it cannot reintroduce the class when wired.

Evidence — a refusal must be distinguishable from an empty cache:

  • X-BR-Cache: miss plus X-BR-Cache-Declined: provider_filter and

X-BR-Cache-Declined-Provider: on both the streaming and non-streaming paths.

  • A policy_gate / cache_declined routing stage, and

decisionTrace.cacheResult = "declined_provider_filter" with a cacheDeclined block, so /v1/explain/{request_id} shows a policy decline rather than an indistinguishable miss.

  • br_semantic_cache_declined_total{reason="provider_filter"}.

The Numbers

  • 5 content-replay read paths audited, 4 filter-aware (the fifth, /v1/replays,

returns routing metadata only — no completion content).

  • 20 new tests. Every negative test was verified to FAIL with the shared

admission predicate neutered (12 failures across both files); the 8 positive controls pass both ways by design, which is what proves the fix did not simply disable caching.

  • 0 additional misses for unfiltered traffic, and 0 for filtered traffic whose

allowlist contains the producer.

Competitive Edge

Portkey and OpenRouter both expose provider allow/deny lists; neither documents what happens when a cached response was produced by a provider the current request excludes. BrainstormRouter now answers that with a recorded fact per cache entry and a header that says which processor was refused — the control is evidenced per request, not asserted in a compliance PDF.

Lockstep Checklist

  • [x] API Routes: no route contract change — response headers only

(X-BR-Cache-Declined, X-BR-Cache-Declined-Provider); site/public/routes.json unchanged.

  • [x] TS SDK: no change required (no request/response body field added).
  • [x] Python SDK: no change required.
  • [x] MCP Schemas: no change required.
  • [ ] Master Record: follow-up — document the two new cache headers in

docs/openapi.yaml, which currently documents no X-BR-Cache-* headers at all.

Follow-up (same day): the exact cache was writing unreadable rows

Live verification of the above on build 62b4c81 showed x-br-cache: miss on four identical unfiltered requests to anthropic/claude-haiku-4-5 — caching looked switched off. It was not a regression from the provenance work (that change is read-side only; the write has no gate on provider, which is optional in both cache tiers). It exposed a separate, older defect.

The prompt cache row is keyed (tenant, promptHash, model). The reader, promptCacheMiddleware, looks up with the model string the CALLER sent, and publishes it on the request context as _promptCacheModel so the writer can use the same key. Nothing read it: non-streaming.ts keyed the row by nsEffectiveModel, the model that actually ANSWERED.

Those differ whenever the requested id is not byte-identical to the registered one. anthropic/claude-haiku-4-5 resolves through isVersionedMatch to the registered claude-haiku-4-5-20251001 endpoint, so the row was written under anthropic/claude-haiku-4-5-20251001 while every lookup asked for anthropic/claude-haiku-4-5. The prompt hash itself commits to the requested model string, so no request could ever produce both that hash AND that key: the rows were unreadable by construction. Every such request paid a provider call and a cache write, forever, and could never hit.

It was invisible because openai/gpt-4o-mini — the model the cache was originally verified on — has a requested id equal to its registered id. Alias and unversioned labels (and auto) silently had no exact cache at all.

Fix: key the write with _promptCacheModel, the exact string the reader used, falling back to the effective model only if the middleware never ran. The routed model is not lost — it is in the cached response body, and provider records who produced it. One consequence worth stating: auto requests now share cache entries with other auto requests carrying the identical prompt hash, which they never could before. That is ordinary exact caching (the hash pins tenant, prompt and the literal model string), it matches what the semantic cache already does for auto, and the provenance check keeps it filter-safe.