Closed a pooled-inference leak: anonymous tenants were billing real model calls to the platform
2026-08-08
LOCKSTEP TRACEABILITY MATRIX --- api_endpoints: ["POST /v1/agents/{agentId}/run"] sdk_methods_updated: ["none — see Lockstep Checklist"] mcp_tools_updated: ["none"] ---
What We Built
A gate that was missing on exactly one route.
POST /v1/chat/completions intercepts tenants with zero provider keys and returns a synthetic brainstorm/sandbox completion, so a newcomer's first call succeeds without touching a real provider. That interception is per-route. POST /v1/agents/{agentId}/run never had it — so the same tenant, on the same key, got real inference from that route while getting a stub from the other, and nothing published explained the split.
Found by the first stochastic-review round (docs/reviews/2026-08-08/), reported independently by two personas, then verified directly against production:
GET /v1/providers -> {"providers":[],"total":0}
POST /v1/chat/completions -> model "brainstorm/sandbox" (correctly stubbed)
POST /v1/agents/{id}/run -> br.model_selected {"model":"o4-mini","provider":"openai"}
agent.message {"text":"42"}
session.status_ended {"totalCostUsd":0.00583508}Correct arithmetic on a tenant with no keys. The credential that served it was the platform's.
Two things made it worse than a routing quirk. The tenant was created by anonymous open signup with no email verification (BRAINSTORMROUTER_OPEN_SIGNUP=1), so no identity was attached to the spend. And the spend was unmetered: the run ledger recorded $0.005835 while /v1/self reported spent_usd: 0 and x-budget-remaining sat frozen at 1.950000 across the whole session. Nothing refused it and nothing counted it.
Why It Matters
This is a real, if small, financial exposure with no attribution: registration is bounded per IP and per day, but within those bounds any anonymous caller could draw on the platform's provider credits, and no budget surface would have shown it.
It also directly contradicts a published claim. llms.txt states _"BYOK — required for real inference; unkeyed sandbox calls return a simulated response."_ That was true of one entry point and false of the other. Fixing the code rather than the sentence makes the documented promise true.
The deeper pattern is the one this codebase keeps meeting: a control implemented per-route rather than at a chokepoint holds only on the routes someone remembered. The sandbox interception was correct, well-commented and well-tested where it existed — and simply absent one file over.
How It Works
The gate refuses rather than simulating:
if (isPostgresConfigured() && (await isTenantInSandboxMode(tenantId))) {
return capabilityResponse(402, {
error: {
message: "Agent runs perform real inference and require your own provider key. …",
type: "sandbox_mode",
code: "provider_key_required",
recovery: { action: "add_provider_key", recoverable: true, endpoint: "POST /v1/providers" },
context: { endpoint: "POST /v1/agents/{agentId}/run" },
},
}) as never;
}Three decisions worth recording.
Refuse, don't stub. The completions stub exists so a newcomer's _first_ call returns 200. The agent loop is multi-turn and executes tools; a canned reply there would invent behaviour an agent could not distinguish from a real run — and an autonomous caller acting on fabricated tool results is worse than one receiving a clean 402. Refusing also matches what llms.txt already promises.
Gate before the session row is written. The check sits after the profile lookup (so an unknown agent still 404s correctly) and before the first getDb(), so a refused run leaves no half-created running session behind.
402, not 403. Already the codebase's idiom for this class — agent-delegate, agents-mesh and liaison/plan all use 402 for budget- and entitlement-shaped refusals.
The refusal is declared in the capability's errors[], so the generated OpenAPI and docs/openapi.yaml now tell callers that a keyless tenant is refused here rather than served — the one place where the asymmetry with /v1/chat/completions is discoverable without reading source.
The Numbers
| Metric | Value |
|---|---|
| Entry points that disagreed on the same tenant | 2 (/v1/chat/completions vs /v1/agents/{id}/run) |
| Verified leaked spend, single run | $0.005835, reported as spent_usd: 0 |
| Reported budget movement | none — x-budget-remaining frozen at 1.950000 |
| Loop entry points needing the gate | 1 — runAgentLoop has exactly one caller |
| Tests added | 3 |
| Unit suite | 9363 passing, 0 failures |
The regression test was confirmed against pre-fix behaviour: with the gate disabled, the request reaches getDb() and the marker fires, proving the run would have proceeded into the loop. A second test asserts the gate is not unconditional — a blanket refusal would pass the first test while breaking every paying tenant.
Competitive Edge
Every gateway meters what its customers spend. The harder property is metering what the platform spends _on their behalf_, on a path nobody was watching, for an identity nobody verified. A governance product that cannot account for its own egress has no standing to attest to anyone else's.
Notably, this was not found by testing. It was found by a persona doing a normal job — onboarding as a new integrator — and noticing that two documented entry points behaved oppositely. That is the argument for the review harness over another test suite: a test asserts what someone already suspected.
Lockstep Checklist
- [x] API Routes:
POST /v1/agents/{agentId}/rungains a declared402
provider_key_required; path, method and request schema unchanged.
- [x] TS SDK: no change required —
packages/sdk-tssurfaces errors generically rather
than enumerating per-route statuses, so the new envelope (type, code, recovery) flows through the existing path. Verified rather than assumed.
- [x] Python SDK: no change required —
_resource.pyraisesBrainstormRouterErrorfor
any status_code >= 400, preserving type and code.
- [x] MCP Schemas: no change — no tool added or altered.
- [x] Master Record: n/a — closes a defect in an existing capability rather than adding one.