Reviewed runs on real hardware. Compare identical workloads and engine revisions.
Run and contribute · Download dataset · Excluded records
Warm request latency is measured after two unmeasured warmups (the first request is retained separately). Each case uses five fixed inputs in three independent worker processes. We take each session's median, then each machine's median, then the median across machines.
Repeated submissions from the same physical GPU and host do not receive additional weight. The interval is a deterministic 95% percentile bootstrap across machines; a single-machine result has no population interval. This is community evidence, not proof against fabricated measurements. Output review and independent reproduction are separate steps.
Driver, operating system, runtime, binary hashes, settings, and the attested power and clock configuration form separate cohorts. CUDA and Vulkan are distinct, and a device nvidia-smi could not attest is marked unattested and pools only with other unattested runs. Raw first-token timing, decode throughput, first-request latency, all outputs, attempts and failures are retained in each evidence bundle.
A campaign refuses to start on a GPU another process is already using, so peak VRAM and power are the whole device's. Sampling runs at a fixed cadence during each measured request; a run whose samples report a hardware or thermal slowdown is retained but not published. These figures are contributor disclosure checked for internal consistency, not recomputed from the outputs the way the timings are. Reference-engine speed ratios are not currently reported.