Observability

Broadcast usage, errors, and memory events to external systems.

Overview

BrainstormRouter's observability engine broadcasts structured events to external destinations in real time. Every API call, guardrail match, quality signal, and memory operation can be forwarded to your analytics stack.

Event types

Type	Fires when
`usage`	Every completion (tokens, cost, latency, model, provider)
`audit`	Memory operations (append, replace, delete, pin, evict)
`quality`	Tool-call accuracy, Thompson sampling rewards
`guardrail`	Content safety matches (PII, jailbreak, policy)
`error`	Provider errors, timeouts, circuit breaker trips

Destination types

Type	Config	Use case
`webhook`	`{ url }`	Any HTTP endpoint
`otlp`	`{ endpoint }`	OpenTelemetry collectors, Langfuse
`datadog`	`{ api_key }`	Datadog Events API
`splunk`	`{ hec_url, hec_token }`	Splunk HTTP Event Collector
`custom`	`{ url }`	Custom HTTP with full control

Setup

// Add a webhook destination
const { destination } = await client.observability.addDestination({
  name: "Slack Alerts",
  type: "webhook",
  config: { url: "https://hooks.slack.com/services/..." },
  event_types: ["error", "guardrail"],
});

// Enable broadcasting
await client.observability.setEnabled(true);

// Test it
await client.observability.testDestination(destination.id);

curl
curl -X POST https://api.brainstormrouter.com/v1/observability/destinations \
  -H "Authorization: Bearer br_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Datadog",
    "type": "datadog",
    "config": {"api_key": "dd_..."},
    "event_types": ["usage", "error"]
  }'

Batching

Configure batch delivery to reduce HTTP overhead:

await client.observability.addDestination({
  name: "Analytics",
  type: "webhook",
  config: { url: "https://analytics.example.com/ingest" },
  batch: {
    max_size: 100, // flush after 100 events
    flush_interval_ms: 5000, // or every 5 seconds
  },
});

Event filtering

Each destination can subscribe to specific event types. A webhook that only receives error events won't get noisy usage events.

Memory events (unique to BrainstormRouter)

Unlike other gateways, BrainstormRouter broadcasts memory operations — when the agent appends, replaces, or evicts a memory entry, that event flows through the observability pipeline. This enables:

Tracking what the agent learns over time
Alerting on PII entering memory
Compliance audit trails via external SIEM
Sleep-time refinement monitoring

Telemetry & EDR integration

Security events from streaming guardrails flow through a structured pipeline to your SIEM/EDR:

%%{init: {'theme': 'dark', 'themeVariables': {'primaryColor': '#d97706', 'lineColor': '#9494a8', 'primaryTextColor': '#e8e8ee'}}}%%
flowchart LR
    SGE["StreamingGuardrailEvaluator\nstreaming-guardrails.ts"] -->|Verdict| Event["Structured Security Event\ntype, severity, action, actor"]
    Event --> Format{"Export Format"}
    Format -->|CEF| CEF["toCef()\nCEF:0|BrainstormRouter|..."]
    Format -->|JSON| ECS["toSiemJson()\nECS-aligned JSON"]
    CEF --> Splunk["Splunk / ArcSight\nQRadar"]
    ECS --> Elastic["Elastic / Datadog\nCustom Webhook"]
    Event --> PG["Postgres\nsecurity_events table"]
    PG --> Batch["exportBatch()\nSeverity-filtered"]
    Batch --> Splunk
    Batch --> Elastic

The SIEM export module (src/security/siem-export.ts) supports two formats:

CEF (Common Event Format) — toCef() produces ArcSight-compatible strings with severity mapping, actor identification, and action details
ECS JSON (Elastic Common Schema) — toSiemJson() produces structured JSON with @timestamp, event.kind, event.category, event.severity, and observer.* fields

Batch export filters by minimum severity (debug=0, info=3, warn=6, error=8, critical=10) so your SIEM receives only actionable events.

Event categories

Event prefix	Category	Example
`auth.*`	authentication	API key validation failure
`config.*`	configuration	Hot config reload
`tls.*`	network	TLS enforcement violation
`guardrail.*`	process	PII detected in streaming output
`schema.*`	change	Schema inference update

Managing destinations

// List all destinations
const { destinations } = await client.observability.destinations();

// Update a destination
await client.observability.updateDestination("dest-uuid", {
  enabled: false,
});

// Remove a destination
await client.observability.removeDestination("dest-uuid");