2026-06-12-inc4-provenance-traceparent

ADR / Ship Log: Full Provenance — Durable Decision Lineage + W3C Trace Context (Increment 4)

Status: PROPOSED (chief-architect synthesis, post-adversarial panel — 2 angles × 6 attack lenses, every fatal/fixable finding resolved or explicitly deferred below)
Date: 2026-06-12
Stacks on: Increment 3 (2026-06-12-inc3-capability-grants-delegation.md), Increment 2 (unified scope enforcement), Increment 1.5 (crash-safe DB-transactional chain head), Increment 1 (scoped grants + denial evidence).
Winning angle: observability-grade substrate (durable decision_lineage table, async bounded writer, zero new synchronous DB round trips, no second hash chain) hardened with evidence-grade's commitment discipline grafted into the existing audit chain: a single hashed provenance_digest field added to the chain payload via the _documented_ schema-version-bump procedure (audit-chain.ts:91–94), plus a synchronous non-hashed policy_context for deny-WHY. The pure evidence-grade angle is REJECTED: its headline tamper-evidence is void as specified (the reused canonicalize is shallow — nested digests hash as {}), its warn-mode second chain is racy under ordinary concurrency, it doubles per-tenant advisory-lock hold with 5–30KB jsonb + WAL inside the critical section, its persist point sheds every non-completions routeCompletion caller, and its own chain needs checkpoint anchoring to resist tail suppression — i.e. a second chain buys strictly less than a 32-byte commitment in the chain we already trust. The pure observability-grade angle is REJECTED as shipped-spec: after it lands, the verifiable tier of an "authorization-and-evidence plane" contains zero WHY, caused_by launders caller-asserted headers into the evidence surface, and deny-WHY — the marquee product moment — rides the lossiest channel in the design while a synchronous durable channel (toolVerdicts/grant_context seam) already exists.

Context

Increment 1–3 made the WHAT verifiable: every completion/denial is a hash-chained completion_audit row; Inc 3 adds verified delegation with grant_context stamping (population lands in their W8). The WHY is ephemeral three times over: the decision-lineage ring buffer (5,000/tenant, decision-lineage.ts:158–174), the routing-event-bus ring buffer (1,000/tenant), and response headers/SSE. The richest WHY (RoutingStage[] @ model-router.ts:283–340, buildDecisionTrace @ :877) never touches the database. The only lineage writer (model-router.ts:799–828) stubs its marquee fields (memories: [], rulesApplied: 0, strategy: "quality" hardcoded) and maintains a private in-memory "chain" that both shares the chainHeads warm-cache Map with completion-audit (audit-chain.ts:239–245) and — because canonicalize (audit-signer.ts:189) is a top-level-keys-only replacer — actually hashes its nested payloads as empty objects. Cross-agent linking is absent: one blind traceparent relay (mesh-capability-routing.ts:149,158), no ingest, no request-id middleware, and a request_id zoo (chatcmpl_, deny_, sandbox UUIDs) in which the router's id (model-router.ts:508) and the audit row's id are never equal — the de-facto join key does not join.

Increment 4 ships two things:

Durable, commitment-bound decision lineage — the rich WHY persisted in Postgres, best-effort for availability but tamper-evident by construction: its digest is hashed into the audit chain at write time, so the WHY can be lost (visibly) but never silently rewritten, and its absence can never be silently manufactured.
W3C trace context — traceparent ingest/mint/store/propagate, with every causal edge labeled by provenance (server_minted vs caller_asserted), composing with Inc 3's authority axis (root_grant_id/chain_subjects) as two orthogonal, mutually corroborating lineages.

One panel finding reshapes the semantics and is stated up front: a hashed 10-field payload plus exclusively non-hashed WHY columns means a privileged DB writer can rewrite every explanation without tripping a single check — grant_context, toolVerdicts, and any new lineage table are all equally mutable. The "never touch the hashed set" stance, treated as law by both proposals, trades away the thesis. The bump procedure exists and is documented; we use it once, for one flat field.

Decision

The WHAT stays in the chain; the WHY is committed _to_ the chain. One new hashed field (provenance_digest, chain payload v2 behind the documented version-bump procedure) binds the deny verdict, the trace identity, and the digest of the rich lineage row into the existing tamper-evident chain at zero additional synchronous DB cost. The rich payload itself lives in a durable, RLS'd, partitioned decision_lineage table written by a bounded async writer. Loss is possible and visible; forgery and silent suppression are not.

Core commitments (each resolves named panel findings):

No second hash chain. Lineage rows carry no prev_hash/event_hash/tenant_seq. Tamper evidence comes from the audit row's hashed commitment over the lineage row's digest. This removes, in one move: the warn-mode tail race (crew-1 A3), lineage tail-suppression and the checkpoint machinery it would demand (crew-0 A3, crew-2 A1 — suppressing a lineage row now leaves a _hashed_ dangling commitment ⇒ verify FAILS, not NULL-ambiguous), the doubled advisory-lock hold and in-lock WAL amplification (crew-1 secondary), and the impossible "stamp lineage_seq after insert" ordering (crew-1 secondary).
Deny-WHY is synchronous, small, and chain-committed. denied/warned outcomes stamp a structured policy_context jsonb on the completion_audit row inside the same logChained insert (the exact grant_context/toolVerdicts seam, CompletionAuditChainedEntry @ completion-audit-store.ts:110) — and its digest is inside provenance_digest. Deny-WHY durability now equals deny-WHAT durability by construction; the async queue carries only bulky routing stages. (Resolves crew-5 A3; corrects the proposal's false claim that denials carry no durable WHY today — recordAuthorizationDenial @ helpers.ts:368+ already stamps reasonCode/violatedScope synchronously; we enrich that channel rather than bypass it.)
A real deep canonicalizer, used only for digests. New deepCanonicalJson (RFC 8785-style: recursive key sort, explicit array handling, fixed number normalization mirroring the toFixed(6) discipline). The existing shallow canonicalize (audit-signer.ts:189) is never handed nested content; the chain payload stays flat — v2 adds provenanceDigest as a top-level hex string, so the flat-payload path is untouched structurally. Negative tests assert nested-field mutation flips verification. (Resolves crew-0 A1 / crew-1 A1, the FATAL finding; also why the in-memory lineage chain is retired, not durabilized — it has this hole live today.)
Edges are labeled by who minted them, decided at write time, committed in the digest. edge_provenance ∈ {server_minted, caller_asserted, none}. verified is reserved for server-minted edges (swarm/agentic/mesh child spans, where BR wrote both rows and created the link) or grant-corroborated edges (post Inc 3 W8). Inbound-header edges from _any_ caller — including authenticated tenant principals — cap at claimed, never asserted-as-evidence; trace_inbound_trusted and the edge label live inside the hashed commitment, so they cannot be upgraded post-write. (Resolves crew-0 A2 intra-tenant grafting, crew-2 A3 flag-flip, crew-5 A2 laundering.)
The detector that can't be opted out of. The authority↔trace cross-check anchors on server-trusted lineage, not attacker headers: when grant_context.depth > 0 and no server-minted or corroborated causal edge exists for the hop, lineage_anomaly: "authority_without_causation" fires — a _missing_ expected edge is the anomaly, so the adversary cannot evade detection by omitting traceparent. Inbound trace context remains corroborating-only and is never an authz/routing/sampling input. (Resolves crew-3 A1.)
One canonical request id, step 0. Middleware mints req_, set on context, plumbed to RouteCompletionParams.requestId (the params.requestId ?? randomUUID() fallback at model-router.ts:508 then yields the canonical id), to all six recordCompletionAudit call sites in completions/index.ts, and to recordAuthorizationDenial (new optional requestId param; standalone denials keep deny_; docblock at helpers.ts:333 updated). Legacy chatcmpl_* ids persist as public_id in the lineage payload. The id-format change is documented for verifiers/dashboards (no chain bump needed — the hash covers whatever value is inserted). (Resolves crew-4 A1: without this, W1 ships a table that joins to nothing.)
span_id exists unconditionally. Minted at capture for every request regardless of BR_TRACE_CONTEXT; the flag gates header _adoption/echo/propagation_, never id existence. span_id NOT NULL makes the (tenant_id, request_id, span_id) idempotency key + ON CONFLICT DO NOTHING actually fire on flush retry. (Resolves crew-4 A2.)
Truncate-then-digest, one provenance object, one write path. The 32KB cap is applied _before_ digesting; the digest is over the exact stored bytes (no false digest_mismatch on the richest rows). A single provenance field on RouteCompletionResult carries {request_id, public_id, trace_id, span_id, parent_span_id, edge_provenance, trace_inbound_trusted, lineage_digest, policy bits} from the router to recordCompletionAudit/recordAuthorizationDenial — audit columns and lineage row derive from one object, so they cannot disagree. The capture block moves after buildDecisionTrace (model-router.ts:877) so the trace exists when captured. (Resolves crew-4 A3, all three sub-failures.)
Coverage does not collapse to the completions route. The lineage write is a first-class store API callable at the router layer. routeCompletion callers that produce no audit row (workflow-runner.ts:18, agents/internal/runner.ts:112, mcp-adapters/routing.ts, intelligence/compare.ts, consensus.ts, embeddings.ts:110, explain.ts:161, register-routes.ts:565,724) get lineage rows with audit_event_hash NULL and a surface discriminator — durable but uncommitted, honestly labeled binding: "unbound" in query results. Completions/denials use the bound flow. Draft lifecycle: drafts are request-scoped objects on the result; unflushed drafts die with the request object — no accumulation. (Resolves crew-1 A2.)
Marquee-field population is an exit criterion, not best-effort. Real strategy, memories, rulesApplied, guardianActive, budgetConstrained (replacing the stubs at model-router.ts:809–820) gate the shadow→on promotion. Durable hollow records are a thesis failure made empirical. (Resolves crew-2 secondary (a), crew-5 secondary.)
Fail-safe to off, everywhere. Lineage capture/writer errors are swallowed with counters in every mode; the writer can never block, delay, or fail a completion or denial. provenance_digest is computable in all modes (commits "nothing was captured" honestly — see threat #7).

Data model + chain-discipline statement

Chain discipline (the one bump, stated precisely)

buildAuditChainPayload gains chain payload v2: the canonical 10 + one flat field provenanceDigest: string (64-hex). Per the documented procedure (audit-chain.ts:91–94): new column completion_audit.chain_schema_version smallint NOT NULL DEFAULT 1, sign-site writes 2 + the field; verify-site dispatches the payload builder by the row's version; v1 rows verify byte-identically forever; mixed-version chains are a first-class test case. The lower-edge probe (evaluateLowerEdge) is version-agnostic and unchanged. Deploy order: verify-side v2 support ships and is verified in prod before sign-side flips (gated by BR_CHAIN_PAYLOAD_V2, default off until verify-side is confirmed; per-row versioning makes rollback safe).

provenanceDigest = sha256hex(deepCanonicalJson({
  v: 1,
  decision_class,                    // "route" | "deny" | "would_deny"
  policy_context_digest,             // sha256 of policy_context bytes, or null
  trace_id, span_id, parent_span_id, // or null
  edge_provenance,                   // "server_minted" | "caller_asserted" | "none"
  trace_inbound_trusted,             // boolean — committed, not flippable (crew-2 A3)
  caused_by_request_id,              // or null
  lineage_digest,                    // sha256 of the lineage row's stored `decision` bytes, or null
  request_fingerprint                // HMAC-SHA256(tenant key, body) — oracle-resistant (crew-0 lesser)
}))

Everything inside the commitment is also stored as plain non-hashed columns for queryability; the commitment makes post-hoc edits to any of them chain-detectable. The commitment struct never touches shallow canonicalize.

`completion_audit` additions (all non-hashed columns; same insert, zero extra round trips; `CompletionAuditChainedEntry` @ `:110` + `logChained` values @ `:210`)

policy_context jsonb — synchronous deny/authority verdict: {reason_code, deny_rule, violated_scope, violating_value, composed_scope_digest, mode, grant_id, root_grant_id, depth, enforced_dimensions}. Populated for denied/warned; null for clean allows (allow-side authority lives in Inc 3's grant_context, pointer-referenced, never duplicated).
trace_id text, span_id text, parent_span_id text, edge_provenance text, trace_inbound_trusted boolean, caused_by_request_id text, lineage_digest text, chain_schema_version smallint.
Docblock at audit-chain.ts:56–70 appends the new names to the EXCLUDED list; v2 section documents the one INCLUDED addition.

New table: `decision_lineage` (schema in `src/db/schema/security.ts`; additive migration; time-partitioned)

decision_lineage (PARTITION BY RANGE (created_at), weekly partitions)
  id bigserial, tenant_id uuid NOT NULL,                  -- RLS: tenantRlsUsing select/insert ONLY (no update/delete policy)
  request_id text NOT NULL,                               -- canonical req_<uuidv7>
  public_id text,                                         -- legacy chatcmpl_*/deny_* id
  audit_event_hash text,                                  -- bound rows: the committed audit row's hash; NULL = unbound surface
  surface text NOT NULL,                                  -- "completions" | "denial" | "workflow" | "mcp" | "swarm_child" | "agentic_step" | "mesh_forward" | ...
  decision_class text NOT NULL, outcome text NOT NULL,
  trace_id text, span_id text NOT NULL,                   -- span_id minted unconditionally
  parent_span_id text, edge_provenance text NOT NULL, caused_by_request_id text,
  selected_model text, selected_provider text, strategy text, reason text,
  decision jsonb NOT NULL,                                -- v-versioned: stages (RoutingStage[], banditCandidates top-8 w/ in-payload truncation
                                                          --   marker), timing struct (decisionTrace DERIVED at read time — not duplicated,
                                                          --   crew-1 secondary), governance{rules_applied,guardian_active,budget_constrained},
                                                          --   memories (ids+hashes, NO factPreview), fingerprint. Cap 32KB, truncate-THEN-digest.
  record_version smallint NOT NULL DEFAULT 1, sampled_rate real,
  truncated boolean NOT NULL DEFAULT false, redacted boolean NOT NULL DEFAULT false,
  created_at timestamptz NOT NULL DEFAULT now()
UNIQUE (tenant_id, request_id, span_id, created_at)       -- partition-key-aware idempotency key (crew-4 A2 note)
INDEX (tenant_id, request_id), (tenant_id, trace_id), (tenant_id, created_at)

Retention = partition drop (no mass DELETE, no BRIN decay — crew-4 secondary), executed by a dedicated retention role; the design states plainly that this role can remove lineage partitions — "append-only" is a policy property of the request path, not a physics claim (crew-3 A3b). Dropped partitions leave dangling lineage_digest commitments by design; verify reports them as expired_by_retention when created_at predates the tenant's retention horizon, missing otherwise.

Exactly one lineage row per (request_id, span_id) per surface (crew-1 rows-per-request ambiguity): the deny verdict is policy_context on the audit row, not a second lineage row; scope/grant composition detail rides inside decision.governance of the single row.

What dies: createDecisionLineage's private in-memory chain and its computeHash/chainHeads coupling (pin test asserts lineage writes never touch the warm cache — crew-2 secondary (d)). The ring buffer survives as an L1 recent-window cache and the off-mode behavior, fed post-enqueue with the same wire shape as the durable read path (one mapper, both modes — crew-4 dual-shape finding). routing-event-bus.ts is untouched (SSE liveness, not evidence).

Capture points (exact files + symbols)

#	Where	Change
0	NEW `src/api/middleware/trace-context.ts`	Mint canonical `req_` + span_id; parse/mint trace context (§traceparent); HMAC request fingerprint computed HERE where the raw body exists (not via re-stringify of `params.messages` in the router — crew-4 secondary); echo `traceparent` + `x-br-request-id` on every response.
1	`src/router/model-router.ts:799–828` → moved after `buildDecisionTrace` (`:877`)	Replace stub block: build the `provenance` object + lineage draft with REAL values in scope — `routingStages`, timing struct, `result.modelId/provider`, actual `effectiveStrategy` (kill the `"quality"` hardcode), savings `source` (`:842`), winning-stage `reason`. Stop calling `computeHash`. Enqueue via `lineage-writer`; attach `provenance` to `RouteCompletionResult`.
2	Marquee-field plumbing into `RouteCompletionParams`	`memories: LineageMemoryRef[]` from the context-injector, `rulesApplied`/`guardianActive` from guardian results, `budgetConstrained` from budget ctx. Explicit plumbing; exit criterion for shadow→on.
3	`src/api/routes/completions/helpers.ts` — `recordCompletionAudit` (`:286` seam) + `recordAuthorizationDenial` (`:368–445`)	Consume `provenance`; stamp `policy_context` (deny verdict: reason_code, deny_rule, composed_scope_digest, mode, grant ids, enforced_dimensions) + trace columns + `lineage_digest` + compute `provenanceDigest` into the v2 payload — all inside the existing single `logChained` insert. Denials during a request adopt the canonical request id (docblock `:333` updated); the Inc 2 `would_deny` site gets identical treatment. Denials always bind — no null-escape-hatch class (crew-2 secondary (b)).
4	`completions/index.ts:641` (`runSwarmCompletion`) + `agentic-completions.ts`	Children inherit `trace_id`, get fresh span_ids, `parent_span_id = parent's span`, `caused_by_request_id = parent request_id`, `edge_provenance = "server_minted"`. `swarmRequestId` survives for budget only.
5	`src/api/capabilities/system/mesh-capability-routing.ts:149,158`	Blind relay → child span: BR re-injects `00---`; enqueues `surface: "mesh_forward"` linking inbound→outbound span (server_minted).
6	Non-completions `routeCompletion` callers (enumerated in Decision)	Unbound lineage rows via the router-level store API, `surface` stamped.
7	Privacy gate at capture (crew-3 A3a)	`privacyRedacted` requests (schema `:214`) ⇒ fingerprint-only payload at the _capture site_: strip `autoSelectMeta.complexity*/confidence`, timing struct, memory refs, savings source — the stripped field list is enumerated in code, not prose. `pii-scanner.ts` runs as a sampled runtime assertion on drafts in shadow/warn, not debug-only (crew-0 lesser).

New module src/security/lineage-writer.ts: bounded queue, sampling decided before serialization (crew-4 secondary), truncate-then-digest at enqueue, flush every 2s/200 records per tenant-group under withTenant, multi-row insert + ON CONFLICT DO NOTHING, retry-with-backoff 60s then drop with br_lineage_dropped_total{reason,kind}. Deny-flood defense (crew-3 A2): repeated identical denials (same principal_id+reason_code+violated_scope+violating_value within a window) aggregate into a counted rollup lineage row (the per-event chained audit rows are untouched — they're the evidence; the rollup is the WHY bulk); per-tenant daily row budget applies to denials with a separate higher reserve; coverage reports the aggregation. Cap accounting is per-replica and documented as approximate (effective cap = N_replicas × budget).

Traceparent design

Ingest (middleware, gated BR_TRACE_CONTEXT=off|ingest|propagate, default off): strict parse ^00-([0-9a-f]{32})-([0-9a-f]{16})-([0-9a-f]{2})$, reject all-zero ids and version ff, 55-char read cap; tolerate higher versions per spec by parsing the version-00 fields. Valid ⇒ adopt trace_id, record caller span as parent_span_id, edge_provenance = "caller_asserted". Absent/invalid ⇒ mint fresh trace_id, edge_provenance = "none". BR's own span_id is always fresh and always exists (flag-independent). Caller's sampled flag is read, stored, never trusted. tracestate stored opaque, 512B truncation, v1 semantics deferred.
Outbound (BR_TRACE_OUTBOUND=off|mesh|providers|all, default off): inject 00--- on mesh forwards (first), provider fetches, tool/webhook/agent-callback egress. Internal children (swarm/agentic) link via caused_by_request_id server-side — no header round-trip.
Joining rule: same tenant + same trace_id = same causal episode (RLS makes cross-tenant grafting structurally impossible); edge B.parent_span_id == A.span_id (cross-service) or B.caused_by_request_id == A.request_id (internal). Edge labels in query output: verified (server_minted, or caller_asserted corroborated by Inc 3 authority: shared root_grant_id / chain_subjects prefix-extension — lights up when their W8 lands), claimed (caller_asserted, well-formed, uncorroborated — regardless of caller auth status), unverified (anything else). Labels derive from the row's own committed fields; chain_subjects display-only (non-hashed jsonb — crew-2 A3). Trace context is never an authz, routing, sampling, or rate input.
Anomalies: authority_without_causation (grant depth>0, no trusted edge — fires on header _omission_, crew-3 A1); digest_mismatch; claimed_edge_contradicts_authority.
Not in scope: no OTel runtime, spans, sampling, or export. otel-gen-ai.ts stays dark; otel-exporter.ts random ids untouched (deferred).

Query surface + lockstep checklist

All defineCapability, permission governance.read, RLS scopedStore, siblings of governance.ts:288/768/988:

Route	Capability	Notes
`GET /v1/governance/lineage/{request_id}`	`system.governance.lineage.get`	The WHY answer: audit row (`verified` block: chain fields + provenance check result) + `policy_context` + lineage row + causal neighbors. `coverage ∈ {captured, sampled_out, dropped, aggregated, pre_rollout, expired_by_retention}` — committed digests make `sampled_out` provable and `dropped` distinguishable from suppression (missing row + hashed commitment + matching drop counters vs. missing row + no honest explanation ⇒ alarm). New `decision-lineage-store.ts`; `requestId` filter added beside `CompletionAuditQueryOpts` (`completion-audit-store.ts:80`).
`GET /v1/governance/lineage?trace_id=…&request_id=…`	existing `system.governance.lineage` (`:988`) re-pointed	Trace reconstruction ordered by span parentage, every edge labeled. One wire shape in all modes (`?live=true` reads the ring buffer through the same mapper).
`GET /v1/governance/audit/chain/verify` (`:768`) extended	—	v2 rows: recompute `provenanceDigest` from stored non-hashed columns + lineage row bytes. Per row: `ok` \	`redacted` (matching chained redaction event exists) \	`missing_lineage` \	`expired_by_retention` \	`failed` (any mismatch, including tombstone-without-event — crew-2 A2).
`explain.ts` narrator	—	Consumes route 1; narration cites committed rows and edge labels.

Lockstep (pre-push hook enforced, same push): TS SDK (packages/sdk-ts: governance.lineage.get/query, trace/request-id response headers typed), Py SDK sync+async, MCP tools (src/mcp/server.ts + site/.well-known/agents.json — agent-facing: "why was I denied" is the product), docs MDX + docs/docs.json (concepts: "Committed lineage — what's provable vs. best-effort"; trace-context spec; coverage semantics; chain v2 note for external verifiers; request-id format change), website positioning, this ship-log → ACCEPTED. llms-full.txt auto on build.

Threat model (attack → defense)

#	Attack	Defense
1	Rewrite the WHY (privileged DB writer edits `decision` jsonb, `policy_context`, trace columns, edge labels)	`provenance_digest` is hashed field 11: any edit breaks chain verify. Deep canonicalizer with nested-mutation negative tests (the crew-0/1 FATAL is closed at the root).
2	Suppress the WHY (delete lineage rows / NULL pointers)	The commitment is in the hashed payload — it cannot be NULLed; missing row vs. committed digest ⇒ `missing_lineage`/`failed`, never silent. Audit-row suppression itself is the existing chain's job (continuity/seq/lower-edge probe, unchanged).
3	Forge a WHY for a real action	Lineage rows without a matching committed digest are flagged; bound rows must hash-match the audit commitment. Unbound-surface rows are labeled `binding: "unbound"` — never presented as committed evidence.
4	Trace grafting — intra-tenant low-priv agent claims a high-trust parent span	Header edges cap at `claimed` regardless of auth (crew-0 A2); `verified` requires server-minted or grant corroboration; labels + trust flag inside the commitment (crew-2 A3). Span ids are disclosed only to the calling principal via response echo. Cross-tenant: RLS-inert.
5	Detector evasion by omitting traceparent	`authority_without_causation` anchors on grant lineage + server-minted edges; the missing edge IS the signal (crew-3 A1).
6	Malicious "redaction" destroys evidence under a privacy excuse	Redaction is itself chained evidence: a `logChained` event (Inc 3 lifecycle-event pattern) recording target, voided digest keys, actor, reason; tombstone without matching event ⇒ `failed` (crew-2 A2).
7	Downgrade — run with lineage off, claim capture	`provenance_digest` commits null lineage_digest/`sampled_out` honestly at write time; an "on"-era audit row with no commitment is impossible; off-era periods are honestly unprovable (inherent to default-off).
8	Deny-flood storage/evidence DoS	Deny verdict is synchronous on the audit row (can't be flooded out of the WHY); lineage bulk aggregates into rollups; per-tenant budgets with deny reserve; 32KB truncate-then-digest; partition-drop retention (crew-3 A2).
9	PII exfiltration via lineage	Structural-metadata-only contract; fingerprint is per-tenant HMAC (no confirmation oracle — crew-0 lesser); `factPreview` excluded from durable record; `privacyRedacted` gated at capture with enumerated stripped fields (crew-3 A3a); retention bounds the window.
10	Writer/DB failure cascades to availability	Writer is fully decoupled from `logChained`; all capture in try/catch; worst case = visible WHY loss, never a blocked request. The only sync additions are header parse, randomBytes, one HMAC, one digest — microseconds, outside the advisory lock's jsonb path.

Rollout + runbook

Flags (wired through the gateway config-reload path — hot, no restart; crew-4 secondary): BR_CHAIN_PAYLOAD_V2 (sign-side gate; verify-side v2 always-on once deployed), BR_LINEAGE_PERSIST=off|shadow|on (default off; shadow = writer persists, reads stay ring-buffer), BR_TRACE_CONTEXT=off|ingest|propagate (default off), BR_TRACE_OUTBOUND=off|mesh|providers|all, BR_LINEAGE_SAMPLE (allows only, default 1.0), BR_LINEAGE_RETENTION_DAYS (30), per-tenant daily caps. Per-tenant overrides follow the Inc 2 dial pattern.
Migration: additive — partitioned table + nullable completion_audit columns + chain_schema_version default 1. v1 rows verify forever; no backfill.
Phases: P0 dark (migrations, middleware mounted with flags off, verify-side v2 deployed + prod-verified on v1 rows) → P1 request-id canonicalization live + shadow writer on internal tenant; gating queries: drop rate <0.1%, digest mismatch = 0, audit↔lineage join rate >99% on captured rows, marquee fields non-stub on 100% of sampled rows (exit criterion) → P2 sign-side v2 on (internal first), ingest trace, query routes + lockstep → P3 on + mesh propagation per cohort → P4 providers/all egress; after Inc 3 W8, claimed edges with grant corroboration auto-upgrade to verified.
Kill switches: BR_LINEAGE_PERSIST=off (queue discarded next tick, ring buffer resumes); BR_CHAIN_PAYLOAD_V2=off (new rows revert to v1; mixed chain verifies by per-row version); BR_TRACE_CONTEXT=off (headers ignored, span minting continues). All ≤ flag-cache latency, no redeploy.
Fail-safe: flag-store outage ⇒ all dials off ⇒ today's exact behavior. Never a request-blocking failure class — lineage shares no transaction with logChained.

Test obligations (adversarial proof set)

Deep canonicalizer property tests: round-trip stability across jsonb insert→SELECT; arbitrary nested structures; negative: single nested-field mutation in stages[i].banditCandidates[j] flips digest verify to failed (the crew-0/1 FATAL pin); number normalization parity sign-site vs verify-site.
Chain v2: mixed v1/v2 chain verifies; v2 row with any commitment-field edit ⇒ failed; lower-edge probe unaffected; sign-flag rollback mid-chain verifies.
Commitment semantics matrix: captured/sampled_out/dropped/redacted/retention-expired × verify outcome; lineage-row delete ⇒ missing_lineage; tombstone without chained redaction event ⇒ failed; with event ⇒ redacted.
Join key end-to-end: completions, streaming, denial, would_deny, swarm child, agentic step, sandbox — one canonical id across audit row, lineage row, response header; legacy public_id preserved.
Idempotent flush: retry-after-timeout produces zero duplicates with span_id NOT NULL; partitioned unique key holds across partition boundary.
Edge labeling matrix: server_minted/caller_asserted/none × authed/anon × grant-corroborated → verified/claimed/unverified; post-write flag-flip attempt breaks chain verify; authority_without_causation fires on header omission with depth>0.
Mode matrix: off = byte-identical current behavior (no schema reads, no writer); shadow = zero read-path change; on; flag-outage = off; hot kill mid-traffic.
Capture coverage: every enumerated routeCompletion surface produces a row with correct surface/binding; privacyRedacted ⇒ fingerprint-only payload (assert stripped fields absent); deny-flood ⇒ rollup rows + budget enforcement, per-event chained audit rows intact.
Isolation pins: lineage path never touches chainHeads (crew-2 (d)); writer failure injection never delays/fails a completion; lock-hold time regression test on logChained (sync additions ≤ digest cost).

Deferred (explicit — no silent drops)

OTel runtime / spans / export — Inc 4 is header plumbing + storage by design; customers with OTel stacks get trace-id interop only. otel-gen-ai.ts stays dark; otel-exporter.ts random-id alignment is a flagged one-liner left out to keep scope honest.
explain.ts traceId rename (helpers.ts:785) — a breaking response-field change needing its own lockstep train; documented as a known landmine in the docs note; new surfaces never use the name.
tracestate semantics — stored opaque/truncated v1; vendor br= entry and mutation rules deferred.
Lineage archival to object storage — partition-drop retention ships now; export-before-drop manifests are named follow-on (the expired_by_retention verify state is the forward hook).
Inbound x-request-id adoption — recorded in payload, never adopted as our key (spoofable join key).
Per-tenant lineage hash chain — REJECTED, not deferred: strictly dominated by the commitment (see Winning angle).
Cross-replica writer coordination / global cap accounting — per-replica loss and approximate caps documented; acceptable at telemetry tier because the evidence tier doesn't depend on the writer.
verified-edge corroboration — mechanically ships now, lights up when Inc 3 W8 populates grant_context; sequencing risk owned by Inc 3.

Honest risks

First chain-payload version bump in the product's history. The procedure is documented but unexercised; a verify-dispatch bug fails verification noisily (fail-safe direction), but external verifiers built on the v1 spec must be migrated via the published v2 test vectors. We accepted this over shipping an evidence product whose every explanation is silently rewritable.
Loss is visible, not prevented. Crash/deploy loses ≤ one flush window of allow-WHY per replica; commitments turn that into auditable dropped counts, not into recovery. Deny-WHY is exempt (synchronous), but rich deny _stages_ under flood degrade to rollups.
Marquee-field plumbing is the schedule risk — cross-module work (memory injector, guardian, budget) gates rollout; if it slips, the increment slips. Chosen deliberately over shipping durable stubs.
Storage 10–30× audit-row growth for sampled-in allows; partitioning + caps + retention manage it, but a 1M-req/month tenant adds 5–30GB/month at sample 1.0.
claimed is the common cross-hop label until Inc 3 W8 — most external causal edges will read as claims, not evidence, in the interim; sales language must match.
jsonb shape rot: decision.stages mirrors RoutingStage; record_version + version-aware readers mitigate, ad-hoc SQL over old rows stays painful.
The request-id format change ripples into any customer dashboard keyed on chatcmpl_* audit ids; public_id + a loud docs/ship-log note are the mitigation, support load expected.
Lockstep tax: two-plus routes, header semantics, and a chain-spec change → full SDK/MCP/docs/website train in one push.

Must implement (independent Opus work items)

W1 — Canonical ids + trace middleware: NEW src/api/middleware/trace-context.ts (mint req_ + unconditional span_id; W3C parse/mint per spec above; HMAC body fingerprint; response echo traceparent + x-br-request-id); plumb to RouteCompletionParams.requestId (fallback at model-router.ts:508), six recordCompletionAudit sites in completions/index.ts (:527,541,555,596,610,623), recordAuthorizationDenial optional requestId + docblock helpers.ts:333. No DB deps. Step 0 — everything joins through this.
W2 — Deep canonical encoder + commitment: NEW src/security/provenance/canonical.ts (deepCanonicalJson, digest helpers, commitment-struct builder). Pure; property + nested-mutation negative tests. No deps.
W3 — Chain payload v2: src/security/audit-chain.ts — versioned buildAuditChainPayload dispatch, provenanceDigest field, docblock updates (:56–70, :91–94 exercised); chain_schema_version column; verify-side dispatch in the chain-verify path (governance.ts:768 consumer); BR_CHAIN_PAYLOAD_V2 gate; mixed-chain tests + published v2 test vectors. Depends: W2.
W4 — Schema + stores: src/db/schema/security.ts — partitioned decision_lineage (RLS select/insert, partition-aware unique key) + completion_audit columns (policy_context, trace set, lineage_digest, edge_provenance, trace_inbound_trusted, caused_by_request_id); migration in src/db/migrate.ts; NEW src/db/stores/decision-lineage-store.ts (withTenant, batch insert ON CONFLICT, request/trace queries); retention job + dedicated role. Depends: nothing (types).
W5 — Async writer: NEW src/security/lineage-writer.ts — bounded queues, sample-before-serialize, truncate-then-digest, per-tenant flush tx, deny rollup aggregator, drop counters, hot-flag wiring through config reload. Depends: W2, W4.
W6 — Router capture: src/router/model-router.ts — relocate capture after buildDecisionTrace (:877), replace stub block (:799–828) with real values + provenance field on RouteCompletionResult; marquee-field plumbing into RouteCompletionParams (memories/rulesApplied/guardian/budget — rollout exit criterion); privacyRedacted capture gate with enumerated strip list; router-level store API for the eight non-completions surfaces; retire createDecisionLineage chain path + chainHeads pin test; ring-buffer feed via the unified wire-shape mapper. Depends: W1, W2, W5.
W7 — Synchronous evidence stamp: src/api/routes/completions/helpers.ts — recordCompletionAudit + recordAuthorizationDenial consume provenance, build policy_context, compute provenanceDigest, extend CompletionAuditChainedEntry (completion-audit-store.ts:110) + logChained values (:210); would_deny parity. Depends: W1–W4, W6.
W8 — Trace egress + internal edges: mesh-capability-routing.ts:149,158 child-span re-injection; runSwarmCompletion (completions/index.ts:641) + agentic-completions.ts server-minted edges; provider/tool-egress injection behind BR_TRACE_OUTBOUND. Depends: W1.
W9 — Query surface + verify extension: governance.ts — lineage.get capability, re-pointed /v1/governance/lineage with trace/request filters + single wire shape, chain-verify provenance check states (ok|redacted|missing_lineage|expired_by_retention|failed), edge-label computation, anomaly fields; explain.ts narrator consumption. Depends: W3, W4, W7.
W10 — Redaction as evidence: chained redaction lifecycle event through logChained (Inc 3 grant_events pattern), tombstone sentinel, verify integration. Depends: W4, W9.
W11 — Lockstep surfaces: TS SDK, Py SDK (sync+async), MCP tools + agents.json, docs (trace spec, coverage semantics, chain-v2 vectors, request-id migration note, runbook), website. Depends: W9 shapes.
W12 — Adversarial suite: the full proof set above as its own deliverable against W2–W10, including the lock-hold regression and writer-failure-injection availability pins.

KILL SWITCH: BR_LINEAGE_PERSIST=off + BR_TRACE_CONTEXT=off revert the request path to today within flag-cache latency; BR_CHAIN_PAYLOAD_V2=off reverts new rows to the v1 payload with per-row-version verify keeping every written segment valid forever.