Quality-aware semantic cache admission — reject broken responses before they get cached

2026-05-20

routercacheintelligence

What We Built

Two new modules that gate the semantic cache:

src/router/cache-admission-gate.ts — evaluateCacheAdmission() rejects responses before semanticCache.store() writes them:

Null content with no tool_calls → null_content_no_tool_calls (reuses the Phase 1 predicate)
Empty string with finish_reason: "length" → empty_string_length_truncation
response_format: { type: "json_object" } requested but body isn't parseable JSON (after stripping common markdown fences) → json_object_invalid_payload

src/router/cache-contract-key.ts — cacheContractKey() returns a short string like "json|m|1" that captures the request's contract shape (response_format type + max_tokens bucket + tools present). Currently exposed for future cache-partition use; the immediate cache wiring uses the admission gate.

Both model-router.ts:cacheStore() and the streaming completion handler (streaming.ts:1239) now pass the request body's response_format/max_tokens/tools so the gate sees the contract context. The metric cache_admission_skipped_total{reason} is emitted on every rejected write.

Why It Matters

OpenClaw stress 2026-05-20 found that 4+ load failures had x-br-route-reason: semantic-cache, cost: 0, similarity: 1.0000 — the same broken response served back over and over because the cache had admitted it. The Phase 0 baseline repro at docs/assessment/openclaw-baseline-2026-05-20/repro/A-null-content.md attempt 3 captures this exact compounding: null-content + length-truncation got cached at attempt 1, returned from cache at attempt 3 with the same null body.

Phase 1 fixed the executor side (throw instead of return broken 200), but a defense-in-depth gate at cache admission stops the next variant of this class from being introduced. The contract key sits ready to add cache partition isolation in a follow-up if json_object requests start crossing into free-text cached responses.

How It Works

// src/router/model-router.ts:cacheStore()
const verdict = evaluateCacheAdmission({
  response: response as CompletionResponseShape,
  request: requestContext ?? {},
  routedModelId: model,
});
if (!verdict.admit) {
  log.info(`Cache admission skipped: model=${model} reason=${verdict.reason} ...`);
  incCounter("cache_admission_skipped_total", { reason: verdict.reason });
  return;
}
// (only good responses reach this point)
this.deps.semanticCache.store(messages, response, model, costUsd, tenantId).catch(() => {});

The gate is narrow on purpose: only known-broken responses get rejected. A free-text response to a free-text request is admitted even if it's mediocre — quality gating beyond contract violations is the bandit's job (Phase 4).

Lockstep Checklist

[x] Internal router change; no API surface change.
[x] SDKs: unaffected.
[x] MCP tools: unaffected.
[x] Ship-log entry (this file).
[x] Tests: 10 unit tests for evaluateCacheAdmission (admit + 3 reject cases + tool-call + json_object variants), 11 unit tests for cacheContractKey (5 partition rules + 6 boundary cases).
[x] Metric: cache_admission_skipped_total{reason} emitted from incCounter().

OpenClaw Plan Reference

Phase 2 of .claude/notes/openclaw-stress-fixes-phased-plan-2026-05-20.md. Stacked on Phase 1 (feat/null-content-cascade). Phases 3-7 to follow.