Home Dashboard GitHub

Streaming Tool Firewall: 7-Check Pipeline, Zero Extra LLM Calls

2026-02-25

tool-call-firewallstreaming-guardrails

LOCKSTEP TRACEABILITY MATRIX --- api_endpoints: ["POST /v1/chat/completions (inline security layer)"] sdk_methods_updated: ["none (transparent to SDK consumers)"] mcp_tools_updated: ["none"] ---

What We Built

Two complementary security systems that operate inline on every LLM response — without making additional LLM calls or adding significant latency.

Tool Call Firewall (tool-call-firewall.ts, 540 LOC): Intercepts tool_calls in LLM responses and runs them through a 7-check validation pipeline. Each tool call receives a verdict (pass/warn/block) with a stable, machine-readable reason code. The firewall supports streaming via StreamingToolCallBuffer, which accumulates delta chunks until a complete tool call can be validated.

Streaming Guardrails (streaming-guardrails.ts, 406 LOC): A stateful token-by-token evaluator that runs on streamed model output. It buffers tokens in a sliding window (20 tokens or until sentence boundary), evaluates for PII, blocked patterns, and governance rule violations, and takes action (allow/truncate/redact/replace) before forwarding to the client.

Both systems are pure pattern matching — no classifier models, no external API calls. This means they add sub-millisecond overhead per chunk and never introduce the latency or cost of an additional LLM round-trip.

Why It Matters

When an LLM decides to call execute_sql("DROP TABLE users") or streams a customer's SSN in a support response, you need to catch it _before_ it reaches the client or executes. Most AI gateways either don't inspect tool calls at all, or rely on a secondary LLM call to evaluate safety — adding 500ms+ latency and additional cost.

BrainstormRouter catches these issues in-line, at wire speed, with deterministic rules that security teams can audit. The firewall's stable reason codes (rbac_unauthorized, command_injection, secret_detected, semantic_anomaly) integrate directly with SIEM systems.

How It Works

Tool Call Firewall — 7-Check Pipeline:

  1. RBAC scope check: Verify the calling principal has tool.execute permission plus tool-specific scopes
  2. Deny list: Match against DANGEROUS_ACP_TOOLS plus custom per-tenant deny lists
  3. JSON parse: Validate that arguments are valid JSON; tag unparseable calls
  4. Built-in arg validation: Pattern matching for command injection (via BuiltinToolArgValidator)
  5. Prompt injection check: Per-leaf validation on all string arguments
  6. Semantic arg policy: Detect destructive patterns per tool type:
  • execute_sql: DROP TABLE, DELETE FROM, TRUNCATE, ALTER TABLE
  • run_command: rm -rf, mkfs, dd if=
  1. Secret redaction: Sanitize API keys, credentials, SSNs via sanitizeToolOutput()

Streaming Guardrails — 3-Layer Evaluation:

Token → Buffer (20 tokens or sentence boundary)
  ├─ Layer 1: Governance rules (highest priority)
  ├─ Layer 2: Blocked pattern regex matching
  └─ Layer 3: PII scanning (activates at 50+ chars accumulated)

Actions per evaluation: allow (forward), truncate (stop stream), redact (replace PII inline), replace (swap with "[CONTENT FILTERED]").

Firewall Modes:

  • block: Enforce verdicts — blocked tool calls are stripped from the response
  • warn: Report via headers but pass through
  • log: Audit trail only, never blocks
  • off: Disabled entirely

The Numbers

  • 7 validation checks per tool call, all pure pattern matching
  • 20-token buffer window for streaming guardrails (configurable)
  • 50-character minimum before PII scanning activates (avoids false positives on short strings)
  • 4 firewall modes — block, warn, log, off
  • 8 stable verdict reasons for SIEM integration: rbac_unauthorized, denied_tool, unparseable_arguments, prompt_injection, secret_detected, command_injection, semantic_anomaly, policy_violation
  • Zero additional LLM calls — pure deterministic pattern matching

Competitive Edge

Portkey has no tool call inspection. OpenRouter has no output guardrails. The few platforms that offer content filtering (Guardrails AI, NeMo Guardrails) require a secondary LLM call, adding 500ms+ latency. BrainstormRouter intercepts tool calls and output tokens mid-stream with deterministic rules, sub-millisecond overhead, and stable machine-readable verdicts. This is the only streaming-aware security layer in the AI gateway market that works without additional LLM calls.

Lockstep Checklist

  • [x] API Routes: Firewall and guardrails are inline middleware on the completions route — no separate endpoints.
  • [x] TS SDK: Transparent to SDK consumers — security verdicts appear as response headers.
  • [x] Python SDK: Transparent to SDK consumers.
  • [ ] MCP Schemas: Not applicable.
  • [x] Master Record: Listed under "Security" in master-capability-record.md.