Orchestration Trajectory Storage — the BrainstormLLM v2 training flywheel

2026-03-28

LOCKSTEP TRACEABILITY MATRIX --- api_endpoints: [ "POST /v1/agent/trajectories", "GET /v1/agent/trajectories", "GET /v1/agent/trajectories/stats", "GET /v1/agent/trajectories/export", ] sdk_methods_updated: [ "client.trajectories.record()", "client.trajectories.list()", "client.trajectories.stats()", "client.trajectories.exportJsonl()", ] mcp_tools_updated: ["br_trajectory_record", "br_trajectory_query"] ---

What We Built

An orchestration trajectory storage system that records the full decision chain of multi-phase agentic workflows. Every trajectory captures which phases ran (spec, design, implement, review, test), which models were selected and how (explicit, Thompson sampling, pattern recommendation), tool usage, token counts, cost, duration, and outcomes.

The system stores trajectories in an in-memory Map (hot cache) with fire-and-forget Redis persistence (90-day TTL). FIFO eviction keeps memory bounded at 10,000 trajectories per tenant. A JSONL export endpoint enables direct push to HuggingFace for training data.

Why It Matters

This is the keystone that unlocks the BrainstormLLM v2 training flywheel. Every orchestrated task run now produces structured training data: what model was chosen for each phase, what tools were used, what the outcome was, and how much it cost. This data can be used to train models that understand optimal orchestration strategies — which model to pick for which phase, when to skip phases, and how to minimize cost while maintaining quality.

How It Works

// Record a trajectory via API
POST /v1/agent/trajectories
{
  "request_summary": "Add user authentication to the API",
  "request_type": "feature",
  "source": "phase-build",
  "project": "my-project",
  "phases": [
    {
      "phase_name": "spec",
      "agent": "product-manager",
      "model": "anthropic/claude-sonnet-4",
      "model_selection_method": "thompson-sampling",
      "tools_used": ["file_read", "grep"],
      "steps_taken": 3,
      "input_tokens": 1500,
      "output_tokens": 800,
      "cost_usd": 0.012,
      "duration_ms": 2500,
      "outcome": "success"
    }
  ],
  "total_cost_usd": 0.057,
  "total_duration_ms": 17500,
  "total_phases": 2,
  "phases_skipped": 0,
  "build_passed": true,
  "tests_passed": true,
  "review_findings": 0
}

// Export for training
GET /v1/agent/trajectories/export
Content-Type: application/x-ndjson

The Numbers

• 4 API endpoints, 2 MCP tools, full TS + Python SDK coverage
• 10 unit tests covering store, query, stats, JSONL export, eviction, Redis
• In-memory hot path with Redis fire-and-forget persistence
• 10,000 trajectories per tenant cap with FIFO eviction
• 90-day Redis TTL for long-term storage

Competitive Edge

No AI gateway competitor (Portkey, OpenRouter, Lasso) captures orchestration-level trajectory data. They track individual requests; BrainstormRouter tracks the full multi-phase workflow that produced those requests. This is the data moat for training models that understand orchestration strategy.

Lockstep Checklist

> _You MUST check these boxes [x] and verify the corresponding files are updated BEFORE committing this log._

• [x] API Routes: src/api/routes/trajectories.ts — 4 endpoints mounted in register-routes.ts.
• [x] TS SDK: packages/sdk-ts/src/resources/trajectories.ts — Trajectories resource with record/list/stats/exportJsonl.
• [x] Python SDK: packages/sdk-py/src/brainstormrouter/resources/trajectories.py — sync + async.
• [x] MCP Schemas: br_trajectory_record and br_trajectory_query in tool-manifest.ts + agents.ts handler.
• [x] Master Record: Trajectory storage is a new capability — update when master record is next reviewed.