Prompts

Template management with versioning, environments, A/B testing, and evaluation.

Endpoints

Method	Path	Auth	Description
POST	/v1/prompts	API Key	Create a prompt
GET	/v1/prompts	API Key	List all prompts
GET	/v1/prompts/:id	API Key	Get with version history
PUT	/v1/prompts/:id	API Key	Update (new version)
DELETE	/v1/prompts/:id	API Key	Archive
POST	/v1/prompts/:id/promote	API Key	Promote to environment
POST	/v1/prompts/:id/test	API Key	Test-resolve with variables
POST	/v1/prompts/:id/ab	API Key	Start/stop A/B test
GET	/v1/prompts/:id/ab	API Key	Get A/B results
POST	/v1/prompts/:id/evaluate	API Key	Run evaluation with test cases
GET	/v1/prompts/:id/evaluate/results	API Key	List evaluation runs
GET	/v1/prompts/:id/evaluate/results/:runId/export	API Key	Export run as CSV or JSON

Create a prompt

curl -X POST https://api.brainstormrouter.com/v1/prompts \
  -H "Authorization: Bearer br_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "id": "support-reply",
    "template": "You are a {{tone}} support agent for {{company}}. {{memory.human}}",
    "variables": {"tone": "friendly", "company": "Acme"},
    "environment": "dev",
    "change_note": "Initial version"
  }'

Request body

Field	Type	Required	Description
id	string	Yes	Unique identifier
template	string	Yes	Template with {{variable}} placeholders
variables	object	No	Default variable values
environment	string	No	dev (default), staging, or prod
change_note	string	No	Description of this version

Update (new version)

PUT creates a new version automatically:

curl -X PUT https://api.brainstormrouter.com/v1/prompts/support-reply \
  -H "Authorization: Bearer br_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "template": "You are a {{tone}} agent for {{company}}. Always be concise.",
    "change_note": "Simplified template"
  }'

Promote to environment

Move a version through dev → staging → prod:

curl -X POST https://api.brainstormrouter.com/v1/prompts/support-reply/promote \
  -H "Authorization: Bearer br_live_..." \
  -H "Content-Type: application/json" \
  -d '{"version": 2, "target_environment": "prod"}'

Test-resolve

Render a template with variables without saving:

curl -X POST https://api.brainstormrouter.com/v1/prompts/support-reply/test \
  -H "Authorization: Bearer br_live_..." \
  -H "Content-Type: application/json" \
  -d '{"variables": {"tone": "professional", "company": "Acme Corp"}}'

{
  "resolved_text": "You are a professional agent for Acme Corp. Always be concise.",
  "template_id": "support-reply",
  "version": 2,
  "is_ab_test": false
}

A/B testing

Start a test

curl -X POST https://api.brainstormrouter.com/v1/prompts/support-reply/ab \
  -H "Authorization: Bearer br_live_..." \
  -H "Content-Type: application/json" \
  -d '{"action": "start", "version_a": 1, "version_b": 2, "split_ratio": 0.5}'

Get results

curl https://api.brainstormrouter.com/v1/prompts/support-reply/ab \
  -H "Authorization: Bearer br_live_..."

Stop a test

curl -X POST https://api.brainstormrouter.com/v1/prompts/support-reply/ab \
  -H "Authorization: Bearer br_live_..." \
  -d '{"action": "stop"}'

Evaluation

Run a prompt against test cases with automatic scoring. Supports template-only evaluation (no model calls) and model-backed evaluation with multiple scoring strategies.

Run evaluation

curl -X POST https://api.brainstormrouter.com/v1/prompts/support-reply/evaluate \
  -H "Authorization: Bearer br_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "test_cases": [
      {"input": {"tone": "friendly", "company": "Acme"}, "expected_output": "You are a friendly support agent for Acme."},
      {"input": {"tone": "formal", "company": "BigCorp"}, "expected_output": "You are a formal support agent for BigCorp."}
    ],
    "scorer": "exact_match"
  }'

Request body

Field	Type	Required	Description
test_cases	array	Yes	Array of test cases (max 50)
version	number	No	Prompt version to evaluate (default: latest)
model	string	No	Model ID for model-backed evaluation
scorer	string	No	exact_match, contains, validity, or llm_judge
store	bool	No	Store results for later retrieval (default: true)

Each test case has:

Field	Type	Required	Description
input	object	Yes	Variable key-value pairs
expected_output	string	No	Expected output for scoring

Scoring strategies

• exact_match — Output must exactly match expected (whitespace-normalized)
• contains — Output must contain expected as a substring
• validity — Structural quality score (0-1) using BR's validity scorer
• llm_judge — A second model call scores output against expected (0-1)

Auto-detection: if no model specified → exact_match. If model specified → validity.

Response

{
  "run_id": "eval_abc123_def456",
  "prompt_id": "support-reply",
  "version": 2,
  "scorer": "exact_match",
  "created_at": "2026-03-22T00:30:00.000Z",
  "test_case_count": 2,
  "pass_count": 2,
  "fail_count": 0,
  "avg_score": 1.0,
  "results": [
    {
      "input": { "tone": "friendly", "company": "Acme" },
      "rendered": "You are a friendly support agent for Acme.",
      "expected": "You are a friendly support agent for Acme.",
      "score": 1.0,
      "pass": true,
      "scorer": "exact_match"
    }
  ]
}

List evaluation runs

curl https://api.brainstormrouter.com/v1/prompts/support-reply/evaluate/results \
  -H "Authorization: Bearer br_live_..."

Returns summaries (without per-case details) for up to 20 stored runs.

Export evaluation run

# JSON export (default)
curl ".../v1/prompts/support-reply/evaluate/results/eval_abc123/export?format=json"

# CSV export
curl ".../v1/prompts/support-reply/evaluate/results/eval_abc123/export?format=csv"

CSV columns: input_vars, expected, actual, score, pass, scorer.