Home Dashboard GitHub

2026-05-20-phase-8-1-2-contract-detect-cache-skip

2026-05-20 — Contract detection + cache skip (Phase 8.1 + 8.2)

Summary

First two steps of the contract-aware-routing arc. Adds:

  • src/router/contract-detect.ts — pure detector. Identifies requests that declare an output contract: response_format, tools[], or literal prompt patterns ("Return only the integer", "Choose one of", etc.).
  • Cache-skip wiring — both the non-streaming (checkSemanticCache) and streaming (pre-stream cache peek) paths in model-router.ts now skip semantic cache when a contract is detected.
  • Response headers — X-BR-Contract-Detected: on every detected request; X-BR-Cache-Skipped-Reason: contract_strict when the contract caused cache to be bypassed.

Why

OpenClaw 2026-05-20 stress caught a real cache-amplification bug that the existing cache-admission gate (Phase 2, #366) can't catch:

  • Gemini Flash Lite returns 345 for (17*19)+23. Correct answer: 346.
  • 345 is structurally valid (non-null, non-empty, parses fine as plain text).
  • Phase 2's gate admits it. Cache stores it. Every identical request gets the wrong answer.
  • All 10 quality-fail samples in the re-run (90/100 vs. 86/100 baseline) traced to this exact cache replay.

The fix-class: when the caller declares a strict output contract, BR should not trust the cache. Future verifier work (Phase 8.3) will add per-entry verifier stamps so contract-strict cache hits are possible again — but ONLY for entries that have been verified. Until that schema lands, the safe default is to bypass cache for contract-strict requests.

What this PR detects

Two confidence tiers:

Machine-readable (high confidence):

  • response_format: { type: "json_object" | "json" | "json_schema" } → kind: json_object
  • tools: [...] declared → kind: tool_args

Literal prompt patterns (lower confidence, recall-biased):

  • "Return only the integer …" / "Output only a number …" → kind: exact_integer
  • Return exactly "..." / Return only "..." → kind: exact_string
  • "Choose one of …" / "Respond with one of …" → kind: enum
  • "Return valid JSON …" / "Reply with only JSON" → kind: json_format

Day-one recall on literal patterns is ~60%. Users phrase the same intent many ways; this catches the unambiguous cases. Everything else falls through to existing behavior — no correctness harm, just no cache-skip benefit.

What this PR is NOT

  • Not task-class inference. Doesn't try to detect arithmetic in free-text prompts. The OpenClaw bug was caught with explicit contract signals; that's the only thing we promise.
  • Not a verifier. Doesn't check whether the response satisfies the contract — that's Phase 8.3.
  • Not a routing change. Doesn't pick different models for contract-strict requests — that's Phase 9+.
  • Not a per-tenant override. Cache-skip on contract is universal. Override mechanism (cache_contract_verified: true) lands with the verifier in Phase 8.3.

Headers

Every detected request, regardless of cache outcome:

X-BR-Contract-Detected: json_object   # or tool_args | exact_integer | exact_string | enum | json_format

When cache was bypassed specifically because of the contract:

X-BR-Cache-Skipped-Reason: contract_strict

Cache-skipped-reason is omitted if the response happened to come from cache anyway (e.g., via a different cache layer like prompt-exact), to avoid misleading observability.

Files

  • src/router/contract-detect.ts (NEW, 220 LOC)
  • src/router/contract-detect.test.ts (NEW, 31 tests)
  • src/router/model-router.ts — cache-skip at both lookup sites + detectContract import
  • src/api/routes/completions/non-streaming.ts — header emission
  • src/api/routes/completions/streaming.ts — header emission

Verification

$ pnpm test:fast
✓ 7556 passed, 0 failed (31 new for contract-detect)
$ pnpm tsgo
✓ no errors

Expected production impact

  • Cache hit rate drops for response_format=json_object requests and any tools[] request. Magnitude depends on traffic mix; OpenClaw's quality phase showed all 10 amplified-failure cases came from contract-strict cache hits, so the drop is mostly draining bad amplification, not good cache value.
  • Provider cost rises slightly for contract-strict requests (no cache means real provider calls). The OpenClaw re-run measured this at ~$0.0003 vs $0.0001 with broken cache — order-of-magnitude smaller than a normal request.
  • Quality of contract-strict responses rises to the freshness of the underlying provider. The 345-vs-346 class of bug stops.

Risks + known limitations

  1. Literal-pattern recall is ~60%. Don't claim universal coverage; downstream users may continue to see wrong answers on patterns we don't detect.
  2. Cache hit-rate dip is unmeasured. The metric will land when observability picks up the new headers. Until then, the only signal is reduced complaints about cache-replayed bad answers.
  3. Prompt-exact cache layer (separate from semantic cache) is NOT affected by this PR. If a downstream layer caches contract-strict responses without verification, the bug class re-emerges there. Should be audited in a follow-up.

Lockstep

  • [x] No API surface change — only adds response headers (additive).
  • [x] SDKs unaffected.
  • [x] MCP tools unaffected.
  • [x] Ship-log (this file).

Plan reference

Phase 8.1 + 8.2 of the contract-aware-routing arc. See locked positioning memory at: ~/.claude/projects/-Users-justin-Projects-brainstormrouter/memory/project_competitive_positioning_2026-05-20.md

Next: Phase 8.3 (verifier layer) — deterministic verifiers for exact string, integer, JSON parse, JSON schema, enum, tool-arg schema. Then 8.4 (reward hook extending Phase 4 bandit V2). Then 8.5 (MCP first demo).