Home Dashboard GitHub

2026-05-20-phase-8-4-verifier-reward-hook

2026-05-20 — Verifier → bandit reward wiring (Phase 8.4)

Summary

Wires Phase 8.3's verifyContract into the bandit reward path so that responses violating a declared contract demote the model in Thompson posterior weights. Also emits X-BR-Contract-Status and X-BR-Contract-Failure response headers on the non-streaming path.

What changed

  1. evaluateOutcomeV2 extended with a new contractVerifier input. New contract_failed classification (validity 0) supersedes the existing JSON heuristic when set, but is correctly ordered below upstream_error / null_content / tool_use_success (hard failures and correct shapes win).
  1. maybeApplyRewardV2 helper (in model-router.ts) now runs detectContract + verifyContract inline when the feature flag (BR_BANDIT_REWARD_V2=1) is on. Verifier failure is non-fatal — wrapped in try/catch so a verifier bug doesn't crash the reward path.
  1. Messages threaded through the helper at the two non-streaming call sites (the third call site, scoreAndRecordValidity, doesn't yet receive messages — documented as a follow-up; enum and exact_string verifiers will be not_applicable from that path).
  1. Header emission added to non-streaming handler: X-BR-Contract-Status: passed|failed|not_applicable and X-BR-Contract-Failure: when the verifier produced a verdict.

What didn't change

  • Streaming handler headers. Streaming responses commit headers before the first byte; verifier can't run inline (response isn't assembled yet). The bandit reward signal still updates (verifier runs in maybeApplyRewardV2 after stream completion), just no Contract-Status header on streaming responses. Could be added as a trailer header in a follow-up.
  • Cache admission gate. Phase 2's gate still only checks structural failures. Hooking the verifier into cache STORE (so contract-failed responses are also rejected at store-time, not just lookup-time) is a clean follow-up but not in this PR.
  • scoreAndRecordValidity messages. That signature accepts tools but not messages — refactoring to pass messages is a follow-up; affects only enum + exact_string verifiers in the audit-chain path.

Decision order in evaluateOutcomeV2 (final)

  1. Upstream 5xx
  2. Executor null-content (cascade already engaged)
  3. Structural null-content / empty-truncation
  4. Tool-use success (content: null + tool_calls is correct shape)
  5. Contract verifier failed ← NEW in Phase 8.4
  6. JSON-contract heuristic (best-effort fallback when verifier wasn't run)
  7. Refusal
  8. Success

Verification

$ npx vitest run --config vitest.unit.config.ts src/router/bandit-reward-v2.test.ts
✓ 39 passed (8 new for Phase 8.4 contract verifier wiring)

$ pnpm tsgo
✓ no errors

Coverage of new path:

  • Verifier failed on usable response → contract_failed (validity 0)
  • Verifier passed and not_applicable are no-ops
  • Upstream_error / null_content / tool_use_success all WIN over verifier failure (priority ordering)
  • Verifier failure supersedes the JSON heuristic
  • Absent contractVerifier leaves V2 behavior unchanged

Files

  • src/router/bandit-reward-v2.ts — new classification + dispatcher branch + VALIDITY_CONTRACT_FAILED constant
  • src/router/bandit-reward-v2.test.ts — 8 new tests
  • src/router/model-router.ts — maybeApplyRewardV2 runs verifier + threads messages; call sites updated
  • src/api/routes/completions/non-streaming.ts — X-BR-Contract-Status + X-BR-Contract-Failure headers
  • Ship-log (this file)

Operational impact

  • Feature-flagged. BR_BANDIT_REWARD_V2=1 (already set per ECS task def for canary; off by default per Phase 4) gates ALL the new behavior. When off, Phase 8.4 is a no-op.
  • Bandit demotion. When V2 is on AND a contract-strict request gets a parseable-but-wrong response, the bandit records validity=0 instead of validity=1. Over time, models that repeatedly fail contracts for a given task class get demoted in the Thompson posterior.
  • No latency cost in the V2-off path. When the feature flag is off, the verifier never runs.

Lockstep

  • [x] No API surface change beyond additive headers.
  • [x] SDKs unaffected.
  • [x] MCP tools unaffected.
  • [x] Ship-log (this file).

Plan reference

Phase 8.4 of the contract-aware-routing arc. Depends on Phase 8.3 (#382). Phase 8.5 (MCP gating) is the next milestone.