SuperWitness is a self-hosted service that joins a SuperPipeline run, its AgentPod attempts, traces, logs and verdicts into one document: what your agents did, how your products behaved, and whether the work was good.
sources: superpipeline ok · agentpod ok · traces ok · logs ok · errors ok · verdicts ok
Example run document. The fields are real; the values are made up.
How it works
At the back, SuperPipeline and AgentPod, then the trace and log stores; at the front, the run's record and the verdict stamp.1
Products send OTLP
SuperPipeline and AgentPod send plain OpenTelemetry to a Collector, which writes it to VictoriaTraces and VictoriaLogs. SuperWitness only queries them. Nothing waits on it.
2
Ask for a run
GET /v1/runs/superpipeline/{board}/{run} joins the run, its attempts with the configuration that ran them, spans, logs and errors when you read it. A source that is down reads unknown, never 0.
3
Record a verdict
A person, a gate or a grader records a verdict. Verdicts are append-only in SuperWitness's own Postgres: a correction is a new verdict. An agent cannot judge a run it executed.
At the back, SuperPipeline and AgentPod, then the trace and log stores; at the front, the run's record and the verdict stamp.
What it's like to use
Say an agent finished a draft on a SuperPipeline board overnight. In the morning you open the SuperWitness app and pick the run from Needs verdict. The trace shows where the time went across the products. Click a tool call and you see what the agent sent and what came back, read live from AgentPod and never stored by SuperWitness. You record a verdict against a rubric, and it stays on the run.
An agent can do the same over MCP, with a bearer token, on the private side of your deployment. The tools the server registers:
get_run
list_run_spans
list_run_logs
list_runs
record_verdict
get_transcript
superpipeline
hub · dispatch
attempt
turn · tool
node-agent
gate · review
run.id = run_7f3a · illustrative
Run it yourself
The latest release is v0.0.8, for Linux and macOS on amd64 and arm64. The installer checks the download againstSHA256SUMS and puts superwitness and its short aliassuwi in ~/.local/bin. It installs no service and starts nothing.
Fake mode serves one development run from built-in fixtures, listens on loopback only, and needs nothing but a Postgres database. It is for looking around, not for real runs.
A real deployment is one Linux host with systemd running SuperWitness next to Postgres, VictoriaTraces, VictoriaLogs and the OpenTelemetry Collector, reading runs from your SuperPipeline and attempts from your AgentPod hub with a service credential. The install guide walks through it.
Where it runs today
It is not standalone yet. It reads runs from SuperPipeline and attempts from AgentPod, and only SuperPipeline runs can be looked up. Working with other agent stacks is a goal, not this release.
It stores verdicts and grader scores; it does not run graders, evaluations or calibration, and it does not compare configurations yet.
Verdicts are append-only, so the service cannot rewrite them. Keeping them is your Postgres backups' job.
No Windows build, no container image and no hosted service. The web app stays off until you allow people to sign in with AgentPod.
Main Street · Witness · from the claims ledger
Claims you can check
Captain Jack's notes, from the lab's claims ledger. Every claim here is one the ledger lets the lab make. Solid means verified against public source. Hollow means true, with a caveat. Open one to see the caveat.
verifiedlimited, caveat attached
One run document joins runs, attempts, traces and logs
Worth knowing. only SuperPipeline runs are addressable. Real use needs SuperPipeline, an AgentPod hub with a service credential, and VictoriaTraces and VictoriaLogs fed by the OpenTelemetry Collector. Not product-agnostic.
Append-only verdicts against versioned rubrics
In short. SuperWitness stores grader scores but does not run graders.
Run registry with filters and latest verdicts
Worth knowing. lists only runs a source reports. Unset SW_RUN_SOURCES means nobody may report.
Web app for reading runs and recording verdicts
Worth knowing. off until SW_ALLOWED_PRINCIPALS names someone. Sign-in goes through AgentPod or the organization plane's OAuth flow and needs an https URL, an OAuth client and a session secret. Allowlisted operators only; not a multi-user SaaS.
MCP server for agents
Worth knowing. the README table lists five and the deployed landing page four; use the code count. /mcp is for bearer-token callers on the private side, not the public edge.
Agent transcripts read live, not stored
Worth knowing. AgentPod only, with transcript routes and a transcripts:read service credential. "Never stored" describes SuperWitness; redaction is AgentPod's and was not audited.
Fake mode to try it without the other products
Worth knowing. one development run with dev: tokens on loopback, still needs Postgres. A demo mode, not stand-alone operation.
Telemetry stays in upstream engines; no OTLP ingest of its own
Self-hosted binary for Linux and macOS, MIT licensed
Worth knowing. the installer installs no service and starts nothing. No Windows build, container image or hosted service; no documented macOS service setup. Installation was not exercised.
Docs checked against code; sites respond
Worth knowing. the check catches names that do not exist, not missing tools, stale versions or wrong prose (see SW-13).
Installs with one command and ships an agent skill
Worth knowing. installs the binary only; the server still needs Postgres and the products it reads. The skill describes operations; it was not evaluated. Not exercised here.
Nothing depends on it, but it isn't standalone yet. No other product needs SuperWitness. Today it reads runs from SuperPipeline and AgentPod; using it with any agent on its own is still on the roadmap.