Home Dashboard GitHub

2026-05-20-openclaw-stress-in-ci

2026-05-20 — OpenClaw quality stress in CI (Phase 6)

Summary

Wires the OpenClaw quality-stress harness into BR's CI as a recurring quality gate. Runs on:

  • Every push to main
  • Nightly at 06:00 UTC
  • Manual dispatch (workflow_dispatch)

Asserts ≥90% quality pass rate across all phases. Initially advisory (continue-on-error: true) for the first 7 stable days; will be promoted to blocking by removing that flag.

Why

OpenClaw's 2026-05-20 stress was a one-shot run that surfaced Phases 1-4's failure classes. Phase 6 turns that one-shot into continuous detection. Every regression of any of these classes — null content, cache contract drift, WAF false positives, bandit reward pollution — will be caught within 24 hours instead of waiting for the next manual stress run.

The 90% threshold matches the per-phase plan target. Cost cap of $1/run protects against runaway provider spend.

How

  • Harness: vendored from ~/Projects/openclaw/workspace/workspaces/main/br_quality_stress.py at this commit. Pinned (no auto-sync). scripts/openclaw-stress/README.md documents provenance and how to update.
  • Workflow: .github/workflows/openclaw-stress.yml. Uses BR_STAGING_API_KEY secret (must be provisioned manually). Falls back to skip-with-warning if secret is missing.
  • Threshold: quality_pass / (quality_pass + quality_fail) across all phases (load, focused, negative if --include-negative).
  • Artifacts: test-results/ uploaded for 14 days for post-hoc analysis.

Security review

Workflow follows defense-in-depth against GitHub Actions injection:

  1. workflow_dispatch inputs (requests, concurrency) are bound to job-level env: vars, then referenced as "$REQUESTS" inside run: — not interpolated directly.
  2. A separate validation step asserts the inputs match ^[0-9]+$ before any shell command uses them.
  3. Secrets are read only via secrets. (no github.event. references).
  4. permissions: contents: read — minimum scope.

Operator prereq

Before this workflow does useful work, set the GitHub Actions secret:

Settings → Secrets and variables → Actions → New repository secret
  Name: BR_STAGING_API_KEY
  Value: <a BR API key with provider credentials, scoped to staging tenant>

Until that secret exists, the workflow logs a warning and exits cleanly (does not fail).

Files

  • .github/workflows/openclaw-stress.yml (NEW)
  • scripts/openclaw-stress/br_quality_stress.py (NEW, vendored)
  • scripts/openclaw-stress/README.md (NEW)
  • Ship-log (this file)

Deferred follow-ups

  • Promote from advisory to blocking after 7 consecutive green nightly runs. Remove continue-on-error: true.
  • Live-mode invariants in nightly — wire pnpm test:invariants with BR_INVARIANT_LIVE=1 into fortress-nightly.yml. Skipped from this PR because a security hook blocks workflow edits as a precaution; would need a separate workflow file or a permission-acknowledged edit pass.

Lockstep

  • [x] No API surface change.
  • [x] SDKs unaffected.
  • [x] MCP tools unaffected.
  • [x] Ship-log (this file).

Plan reference

Phase 6 of .claude/notes/openclaw-stress-fixes-phased-plan-2026-05-20.md. Depends on Phases 1, 2, 3 (all merged). Sets up the recurring detection layer for Phase 7's verification.