R36 sprint: 4 PRs close 8 fortress fails, lift /v1/ops/status capacity signals
2026-05-21
What We Built
Four PRs landed in one session, all targeting the panel-cited risks from R35's stochastic assessment:
- #391 — ErrorEnvelope canonicalization on 401/403/429 + return 404 (not 503) for unknown models. Closed R35's universal 10/10 panel risk.
- #393 — Embeddings endpoint normalizes bare OpenAI model names (
text-embedding-3-small→openai/text-embedding-3-small). Closes EMB-001/002/003 code-side; upstream provider key still needs embeddings access. - #394 — RBAC middleware returns HTTP 405 with
Allowheader when path exists for another method (was 403 masking 404). HTTP RFC 9110 §15.5.6 compliance + matches OpenAI behavior. Closes ERR-005. - #395 —
/v1/ops/statusaddscapacityfield with process_uptime_seconds, heap_used_mb, heap_total_mb, rss_mb. Closes R35's 3/10 capacity-signal risk.
Why It Matters
The R35 panel unanimously flagged ErrorEnvelope drift as the #1 blocker. A single PR closed it, then three follow-ups tightened adjacent surfaces. End-to-end loop in one session: PR → CI → merge → ECS deploy → SHA-verified via /v1/.well-known/build.json polling → fortress re-run → result captured.
Fortress count moved from R34/R35's 175 pass / 13 fail to R36's 180 pass / 8 fail (best stable run, before credential-stuffing protection kicked in from heavy test traffic).
How It Works
The four PRs share a common pattern: each closes a specific panel-cited risk via narrow, focused code change with verifiable before/after behavior.
ERR-005 example:
// src/api/middleware/rbac.ts
const allowedMethods = findAllowedMethodsForPath(c.req.path);
if (allowedMethods.size > 0) {
const allow = [...allowedMethods, "OPTIONS"].sort().join(", ");
c.header("Allow", allow);
return c.json({ error: { ... type: "method_not_allowed" }}, 405);
}
// fall through to 403 mask for truly unmapped paths
Verified live:
$ curl -X GET https://api.brainstormrouter.com/v1/chat/completions
HTTP 405 Allow: OPTIONS, POST
The Numbers
| Metric | R35 | R36 (post-fix) | Δ |
|---|---|---|---|
| Fortress pass count (best run) | 175 | 180 | +5 |
| Fortress fail count | 13 | 8 | -5 |
| Unknown-model HTTP code | 503 | 404 | (RFC-correct) |
| Wrong-method HTTP code | 403 | 405 | (RFC-correct) |
| ErrorEnvelope on auth errors | partial | canonical | full shape |
| /v1/ops/status capacity signals | absent | present | new field |
Competitive Edge
Portkey, Helicone, LiteLLM all return varying error shapes; none have a published canonical ErrorEnvelope schema with code/request_id/recovery fields. BR's structured-error contract gives agent consumers retry-classification primitives that aren't available elsewhere.
Lockstep Checklist
> Verified before commit:
- [x] API surface: 405 status code is additive (was 403, now more specific).
Allowheader is new but RFC-required. - [x] SDK: no method changes — only HTTP status codes shifted. Existing SDK error-handling already covers 4xx.
- [x] MCP: no tool changes.
- [x] Ship-log: this file.
Next session targets (toward goal 95)
- Fortress test determinism (6/10 panel risk — cross-test pollution)
- Real Scale Readiness lift (load test at 50/100 concurrent, ECS autoscaling validation)
- Brainstorm integration (BSM-001/006), Budget governance (BGV-001/005), Moderations (MOD-003) — separate per-bug investigation