Every run, on the record.

SuperWitness is a self-hosted service that joins a SuperPipeline run, its AgentPod attempts, traces, logs and verdicts into one document: what your agents did, how your products behaved, and whether the work was good.

run_7f3a superpipeline · draft
GET /v1/runs/superpipeline/brd_01/run_7f3a
agent
hermes 0.9 · sha256:4dd8…
attempts
2 · completed
trace
joined · 41 spans
errors
0
cost
unreported
verdicts
gate: approved · human grader 0.8

sources: superpipeline ok · agentpod ok · traces ok · logs ok · errors ok · verdicts ok

Example run document. The fields are real; the values are made up.

How it works

  1. At the back, SuperPipeline and AgentPod, then the trace and log stores; at the front, the run's record and the verdict stamp.
    1

    Products send OTLP

    SuperPipeline and AgentPod send plain OpenTelemetry to a Collector, which writes it to VictoriaTraces and VictoriaLogs. SuperWitness only queries them. Nothing waits on it.

  2. 2

    Ask for a run

    GET /v1/runs/superpipeline/{board}/{run} joins the run, its attempts with the configuration that ran them, spans, logs and errors when you read it. A source that is down reads unknown, never 0.

  3. 3

    Record a verdict

    A person, a gate or a grader records a verdict. Verdicts are append-only in SuperWitness's own Postgres: a correction is a new verdict. An agent cannot judge a run it executed.

At the back, SuperPipeline and AgentPod, then the trace and log stores; at the front, the run's record and the verdict stamp.

What it's like to use

Say an agent finished a draft on a SuperPipeline board overnight. In the morning you open the SuperWitness app and pick the run from Needs verdict. The trace shows where the time went across the products. Click a tool call and you see what the agent sent and what came back, read live from AgentPod and never stored by SuperWitness. You record a verdict against a rubric, and it stays on the run.

An agent can do the same over MCP, with a bearer token, on the private side of your deployment. The tools the server registers:

  • get_run
  • list_run_spans
  • list_run_logs
  • list_runs
  • record_verdict
  • get_transcript
superpipeline
hub · dispatch
attempt
turn · tool
node-agent
gate · review
run.id = run_7f3a · illustrative

Run it yourself

The latest release is v0.0.8, for Linux and macOS on amd64 and arm64. The installer checks the download againstSHA256SUMS and puts superwitness and its short aliassuwi in ~/.local/bin. It installs no service and starts nothing.

$ curl -fsSL https://github.com/SuperJackfruitLabs/superwitness/releases/latest/download/install.sh | bash

First command

$ SW_FAKE_SOURCES=1 SW_DATABASE_URL='postgres://<user>:<password>@127.0.0.1:5432/<database>?sslmode=disable' superwitness serve
$ curl -H 'Authorization: Bearer dev:prn_human01:human' localhost:8790/v1/runs/superpipeline/brd_01/run_01

Fake mode serves one development run from built-in fixtures, listens on loopback only, and needs nothing but a Postgres database. It is for looking around, not for real runs.

A real deployment is one Linux host with systemd running SuperWitness next to Postgres, VictoriaTraces, VictoriaLogs and the OpenTelemetry Collector, reading runs from your SuperPipeline and attempts from your AgentPod hub with a service credential. The install guide walks through it.

Where it runs today

  • It is not standalone yet. It reads runs from SuperPipeline and attempts from AgentPod, and only SuperPipeline runs can be looked up. Working with other agent stacks is a goal, not this release.
  • It stores verdicts and grader scores; it does not run graders, evaluations or calibration, and it does not compare configurations yet.
  • Verdicts are append-only, so the service cannot rewrite them. Keeping them is your Postgres backups' job.
  • No Windows build, no container image and no hosted service. The web app stays off until you allow people to sign in with AgentPod.