Stacks on: Increment 1.5 (feat/inc1.5-audit-crash-safety).
Full design output (read for the complete must-implement + ADR prose):/private/tmp/claude-501/-Users-justin-Projects/fb398c2a-7b23-4c95-84bc-a58fea0f6bb3/tasks/w9l8achf8.output
Context
BrainstormRouter models authority in many places but enforces almost none of it. Increment 1 enforced ONE dimension (the liaison plan's allowed_models/allowed_tools) at a guardian chokepoint + router execution boundary, with denials in a tamper-evident chain (Inc 1.5 made the chain crash-safe). Increment 2 wires the OTHER modeled-but-unenforced scopes onto that SAME enforce+evidence pattern — killing the "modeled-not-enforced" liability class — but only the sources that are trustworthy to enforce and tractable now.
Decision
You may only enforce a scope you can cryptographically trust, and only at a dimension it actually expresses. Increment 2 enforces the model dimension via a fail-closed composer.
In scope
api_key.allowedModels (DB-trusted) — model dimension.
Liaison plan allowed_models/allowed_tools (already enforced) — folded into the composer, no behavior change.
Trust-envelope br_scope.models (Ed25519-signed by BR) — enforced via the composer at the boundary, NOT via the silent candidate filter.
Deferred (with reasons)
agent-JWT scopes — HMAC-only; already an RBAC/tool-auth input via isToolAuthorized; it's a coarse role gate, NOT a model/tool allow-list. Don't repurpose.
behavior-cert resource_scopes — HMAC + in-memory store + dev-only fallback key + checkResourceScope fails OPEN + wrong dimension (resource type/path, not model/tool) + no request-path call site.
delegation act_chain.scope — UNSIGNED string[] (no signature, no verify). Enforcing it would be a privilege-escalation primitive. Requires Increment 3 (cryptographic delegation).
api_key.allowedIps — no inet matcher; needs an XFF/proxy-trust decision.
Tier 1 (coarse, unchanged): RBAC hasPermission(principal,'router.write') to route at all; isToolAuthorized gates tools. RBAC's legacy "no scopes = allow" stays strictly here.
Tier 2 (the new composer): effective model allow-list = deny-wins INTERSECTION over all PRESENT fine-grained sources, each normalized to scope-match.ts algebra at the type boundary. Per-source normalization: SQL null/absent ⇒ undefined (no restriction, SKIP); present-but-empty [] ⇒ deny-all (NEVER mapped to skip); ['']/br_scope '' ⇒ affirmative unrestricted. Permitted iff matchCapabilityScope succeeds against EVERY present source. Monotone non-widening: a source may only TIGHTEN, never widen, a more-trusted source (composed set ⊆ every input). Trust ranking: Ed25519 envelope / DB api-key > HMAC agent-JWT (narrow-only) > HMAC behavior-cert > unsigned act_chain (never authorizes). Final decision = RBAC_coarse_gate AND (executed label ∈ intersection).
Documented exception (rollout-safety): during _warn_ phase, a present-but-empty api_key.allowedModels [] is treated as legacy-permissive and emits a would_deny row tagged empty_allowlist_legacy; it only becomes hard deny-all at enforce-flip. Single documented exception to deny-by-default, scoped to api_key, to avoid a guaranteed multi-tenant outage on legacy keys.
Rollout mechanism
Per-source off|warn|enforce tri-state, default = off = exact current behavior.[^community-pregate] resolveScopeMode(source, tenantId): a feature-flag (src/infra/feature-flags.ts) gates rollout-cohort membership (one flag per source: enforce_apikey_models, enforce_envelope_models); tenant-settings security.scopeEnforcement. ∈ {off,warn,enforce} is the operator dial (same place guardian mode + toolFirewall.mode live). Both absent ⇒ off ⇒ legacy. feature-flags fail-closed = fail-SAFE here (reverts every tenant to legacy-permissive, never a mass-403) — MUST be documented so nobody "fixes" it to fail-open. warn records-then-proceeds (would_deny evidence, blocks nothing); enforce records-then-denies (403). Kill switch: add tenantId to the flag's tenants_blocked (wins over all precedence) ⇒ reverts to off within the 60s cache cycle, no redeploy. Flip runbook per source: deploy off → canary warn (tenants_allowed) → run would_deny + stale-id audit → fix offending keys → flip tenant setting to enforce → ramp via rollout_percent.
[^community-pregate]: Exception: the community-tier pre-gate (widens-only, applies only to tier === "community" keys) is NOT mode-gated — see the Implementation-notes correction (G5 follow-up #6).
Must implement (core — read the full design output for all items)
Create src/security/effective-scope.ts: composeModelScope(sources: {origin:'api_key'|'plan'|'br_scope'; allowed: readonly string[] | undefined}[]): string[] and composeToolScope(...). Deny-wins intersection over PRESENT sources via scope-match.ts algebra; undefined=skip, []=deny-all, ['*']=unrestricted; output normalized; guarantee composed ⊆ every present input (monotone non-widening).
Re-export TrustGateMode as ScopeEnforceMode from scope-match.ts; add resolveScopeMode(source, tenantId) (feature-flag cohort + tenant-settings, default 'off').
Normalize api_key.allowedModels at the type boundary BEFORE composing: SQL null ⇒ undefined (NEVER []); present [] ⇒ deny-all (warn-phase legacy-permissive empty_allowlist_legacy); canonicalize each entry via canonicalGrantLabel (db labels may be bare modelId OR provider/modelId); replicate synth.ts:193apiKey.allowedModels ?? "*" exactly.
Compute effectiveAllowedModels = composeModelScope([api_key, plan, br_scope]) in completions/index.ts (~1291) and thread into RouteCompletionParams.allowedModels at BOTH streaming.ts:144 and non-streaming.ts:170. Force route.allowFallback=false when ANY source present (currently only when plan present).
DELETE the raw apiKey.allowedModels.includes check at index.ts:1103 (subsumed by composer; fails open on [], breaks on ['*']). Audit community-tier.ts:110 and prompt-cache.ts:191 (same pattern) → route through composer or remove; ONE authoritative recording site per request (Inc1 one-chokepoint discipline).
Fix the routing-gates fail-open: model-router-select.ts:650-665 — do NOT gate filtered = gateResult.candidates on gateResult.source!==null. Hard-enforce br_scope.models ONLY through composer+assertModelInGrant boundary so an empty intersection throws ModelScopeError (recorded), not a silent empty pool.
Promote enforcePlanScopeModeGate (handler chokepoint at index.ts:485 before agentic/swarm/sandbox) to enforceComposedScopeModeGate consuming the composed grant; gate-on-presence = "any non-wildcard composed grant present" (not just committed plan). sandbox ⇒ grant-check SANDBOX_MODEL via composed set; agentic/swarm ⇒ reject under any active model-scope source (scope_mode_unsupported). Add composed grant to the cache-path guard.
Add wouldDeny?: boolean to recordAuthorizationDenial (helpers.ts:348): warn-mode records a non-blocking marker WITHOUT changing hashed chain bytes (mode/violated-source in the non-hashed columns).
Test obligations
Composition: intersection of api_key+plan+br_scope (most-restrictive binds); monotone non-widening property test; null=skip / []=deny-all / ['*']=unrestricted; canonical-label normalization. Bypass: out-of-composed-grant model on every path (incl agentic/swarm/sandbox/cache, streaming+non-streaming, executed-model boundary). Rollout: off=legacy (no behavior change), warn=records would_deny but allows, enforce=403; feature-flag outage ⇒ fail-safe to legacy; tenants_blocked kill switch; empty_allowlist_legacy warn-phase exception. Evidence: each denial records the violated source+rule; warn would_deny rows don't corrupt the hash chain.
Implementation notes (as shipped)
Composer — src/security/effective-scope.ts: composeModelScope / composeToolScope are a deny-wins INTERSECTION over PRESENT sources (undefined=skip, []=deny-all short-circuit, ['']=identity element). Output is a valid scope-match.ts allow-list ([''] ⇒ nothing restricted, [] ⇒ deny-all, else the normalized intersection). Guarantee proven by a pair+triple property test: matchCapabilityScope(composed, x) ⇒ matchCapabilityScope(s, x) for every present source s (composed ⊆ every input — monotone non-widening). Normalizers normalizeApiKeyModels (SQL null⇒undefined, []⇒[], canonicalize entries) and normalizeBrScopeModels (string ''⇒['']) live here too.
Mode resolution — resolveScopeMode(source, tenantId) in scope-match.ts (ScopeEnforceMode re-exported there). Order: per-source flag cohort (enforce_apikey_models / enforce_envelope_models) → tenant-settings security.scopeEnforcement.. DEFAULT-OFF / FAIL-SAFE: a flags outage (fail-closed=false) or any settings-read failure resolves to off = legacy-permissive, NEVER a mass-403. This is load-bearing — do not "fix" the fail-closed flag default into fail-open.
Chokepoint — evaluateComposedModelScope (completions/composed-scope.ts) is the ONE authoritative recording site, replacing the raw .includes at index.ts:1103. The enforced composed grant is threaded into RouteCompletionParams.allowedModels (via the existing planAllowedModels ctx field, which also drives the streaming cache-bypass guard) so assertModelInGrant re-checks the EXECUTED model after BYOK/budget swaps. Short-circuit paths covered by enforceComposedScopeModeGate (gates on "any non-wildcard composed grant present"). community-tier.ts / prompt-cache.ts now route their admission checks through matchCapabilityScope (correct ['*']/[]), no longer authoritative recording sites. The model-router-select.tssource!==null fail-open is fixed (scope filter always applies in enforce mode).
> Correction (Inc2.1, G5 follow-up #6): the community-tier gate is canonical in ALL modes — it is NOT mode-gated (it never calls resolveScopeMode) and therefore NOT byte-exact legacy in off. The delta is widens-only (['*'] now correctly unrestricted; entries trimmed/canonicalized; present-[] defers to the handler's rollout decision) and applies ONLY to tier === "community" keys; the authoritative, mode-faithful enforcement remains the handler chokepoint, which still runs after this gate. (prompt-cache.ts, by contrast, DOES mode-gate via apiKeyModelDecision and stays off/warn legacy-faithful.)
Evidence taxonomy (NON-hashed reasonCode + pure-dimension violatedScope): in-use apikey_model_scope_exceeded, envelope_scope_exceeded, scope_mode_unsupported, empty_allowlist_legacy (+ existing plan_scope_exceeded); RESERVED-not-emitted delegation_scope_exceeded, cert_resource_scope_exceeded. Warn would_deny rows use outcome "warned" (distinct from enforce "denied") — a self-consistent, verifiable chain link that does not corrupt the chain.
Deferred sources (NOT enforced — explicit)
agent-JWT scopes (HMAC, RBAC-shaped), behavior-cert resource_scopes (HMAC + in-memory + dev key + fail-open matcher + wrong dimension), delegation act_chain.scope (UNSIGNED — enforcing it is a privilege-escalation primitive; HARD DEFER to Inc3), api_key.allowedIps, and the unpopulated br_scope.providers/tools/regions stubs. Reserved reason-codes register the deferred slots so no later contributor "finishes the wiring" and opens the act_chain self-assertion hole.
Post-G1 review fixes (3 findings)
Three review findings on this increment were resolved (all tests green; one commit per finding):
Cache-admission composed-scope bypass (P1). The prompt-cache middleware (prompt-cache.ts) returns a same-tenant HIT BEFORE the handler composer + router boundary evaluate the request's composed model scope. The admission guard only consulted api_key.allowedModels, so an enforce-mode trust-envelope br_scope.models could be bypassed by a cached response from an unrestricted request (same class as Inc1's semantic-cache bypass). FIX: cache admission now composes the ACTIVE scope — api_key (always-on, every mode, matching the base raw .includes) plus br_scope (only in enforce mode, since br_scope has no legacy block) — and refuses a hit the composed grant would not authorize, flowing the request to the authoritative 403. off/warn br_scope leave cache behavior unchanged (legacy). A present plan already forces bypassCache upstream, so it is not re-derived here.
Enforce mode wrongly consulted the legacy raw .includes (P2). In composed-scope.ts the legacy .includes ran before the canonical apiKeyNorm decision in ALL modes, so a valid bare grant ["gpt-4o"] requesting openai/gpt-4o was FALSE-denied even though normalizeApiKeyModels(..., resolveBareId) had expanded it to the canonical label. CORRECTED 3-mode invariant (now stated explicitly in the INVARIANT (DO NOT RE-BREAK) comment):
off ⇒ the LEGACY decision governs ALONE (exact base .includes: byte match, []=deny-all, no canonicalization). Zero behavior change.
warn ⇒ the legacy decision is a FLOOR (legacy .includes deny ⇒ 403, preserving pre-existing behavior); a NEW-only-stricter denial (legacy allows, canonical denies) ⇒ record would_deny + allow.
enforce ⇒ the CANONICAL apiKeyNorm decision governs ALONE; the legacy raw .includes is NOT consulted (bare-id resolution, ['*'], []=deny-all + evidence). For an auto request the canonical membership test defers to the boundary (it was previously a legacy 403).
The empty_allowlist_legacy exception (off=deny-all, warn=permissive would_deny, enforce=hard deny-all) is unchanged. Test updated to encode the corrected enforce behavior: enforce api_key + brainstormrouter/auto now DEFERS to the boundary (canonical alone, grant threaded) instead of a legacy 403; all off=legacy and warn-floor tests are unchanged and still green. Added Finding-2 3-mode parity tests (enforce + bare [gpt-4o]→openai/gpt-4o ⇒ allow, enforce + ['*'] ⇒ allow, off + bare ⇒ legacy deny, warn + bare ⇒ legacy floor 403).
Warn-mode AUTO selections recorded no would_deny evidence (P2) — FIXED (not documented). A warn-mode br_scope / api-key-new-strictness auto request defers its concrete membership check pre-router (the executed endpoint is unknown until routing), but warn grants were NOT threaded onward, so the boundary neither threw nor recorded — a warn canary missed every auto request that WOULD deny after the enforce flip. FIX (the preferred boundary-observe path): the chokepoint now emits observeSources (warn-mode grants, auto requests only — concrete warn misses are still recorded inline pre-router, so no double-recording), buildObserveGrantParams composes them into an observeModels grant + an onObserveWouldDeny callback, and these thread through RouteCompletionParams.observeModels / onObserveWouldDeny into both router paths. At the EXECUTION BOUNDARY (alongside assertModelInGrant, downstream of BYOK/budget swaps) the new non-throwing observeModelGrant evaluates the ACTUALLY-executed model and fires the callback, which records ONE non-blocking would_deny (recordAuthorizationDenial({ wouldDeny: true })) per violated source with the right reasonCode. The router stays free of audit deps (the callback closure, built in the handler, owns the recorder). CONCRETE-model warn would_deny continues to be captured pre-router. Covered by new unit tests (composed-scope.test.ts observe-source collection + buildObserveGrantParams; model-router-select-observe.test.ts boundary helper).
G5 follow-ups (Inc 2.1)
The G5 jury approved Increment 2 (9/9) with NON-blocking follow-ups. Items #3, #5, #6 are docs/tests/comments only (no behavior change — see the community-tier correction above and the ACCEPTED GAP comment in composed-scope.ts). Item #4 is the ONE behavior-changing follow-up:
#4 — Cascade deny-all divergence (enforce / plan-deny-all only).filterCascadeTiersByGrant (model-router-cascade.ts) used to pass a PRESENT empty grant [] through unchanged (|| grant.length === 0), unlike the auto pool (computeOutOfGrantExclusions) and multi-model pool (filterEndpointPoolByGrant) which treat [] as deny-all. Removed that clause so [] now falls into the restricting branch ⇒ no tier survives ⇒ a recorded ModelScopeError thrown pre-execution by BOTH tryCascadeCompletion and tryCascadeStream, naming tier-0. [] reaches allowedModels ONLY from enforce-mode sources or a deny-all plan (off/warn never fold a [] grant into the composed allow-list), so off/warn byte-exactness is unaffected. Observable deltas (enforce / plan-deny-all [] only): (1) the prior 404-no-evidence edge — tier-0 with no registered endpoints under strict mode — becomes a recorded 403 (strictly better); (2) denied-model attribution is tierGrantLabel(tiers[0].model) (usually the same string the boundary would have produced); (3) tier-0 resolution side effects (routing stages, circuit-breaker touches, logs) no longer occur. Covered by cascade tests (deny-all [] ⇒ ModelScopeError naming tier-0, zero routeCompletion calls, both paths; ['*']/absent ⇒ pass-through; partial grant ⇒ only in-grant tiers).
Deploy with the flag absent ⇒ every tenant resolves off ⇒ exact legacy behavior (zero behavior change).[^community-pregate]
Canary a tenant: add it to the flag's tenants_allowed AND set security.scopeEnforcement. = warn. Warn emits would_deny rows (outcome warned) and blocks nothing.
Audit before enforce — run BOTH gating queries and resolve every finding:
would_deny query: SELECT tenant_id, request_id, model, tool_verdicts FROM completion_audit WHERE outcome = 'warned' AND created_at > now() - interval '7 days'; (the tool_verdicts JSONB carries reasonCode / scope: / value: / mode:warn). Any rows ⇒ a key whose allowlist would 403 under enforce — fix the key (or notify the customer) FIRST.
stale-id audit: for each active key with non-null allowed_models, resolve every entry through canonicalGrantLabel against the live catalog and flag entries resolving to ZERO live endpoints — these are the stale ids that 403 on flip.
Enforce — set security.scopeEnforcement. = enforce for the audited tenant. Denials now record outcome denied AND return 403.
Ramp via rollout_percent (stable per-tenant hash ⇒ a tenant never flaps).
KILL SWITCH: add the tenantId to the flag's tenants_blocked array — wins over all precedence, reverts that tenant to off within the 60s cache cycle, NO redeploy.