An interactive, password-protected Vercel dashboard for exploring AgentX serving sweeps. It plots per-GPU output-token throughput against per-user interactivity by default, computes the non-dominated frontier from the currently filtered accepted measurements, and keeps every other configuration visible for comparison.
This first dataset covers the Kimi-K3 MI355X work:
- 35 TP8/TEP8 × KV-offload screen attempts across concurrency 1, 2, 4, 8, 16.
- 31 accepted screening measurements and 4 pending cells.
- 7 concurrency-1 attention, all-reduce, and memory-control diagnostics.
- Complete filters for parallelism, KV mode, attention backend, all-reduce dispatch, GPU memory utilization, concurrency, run type, and validity.
The design and data model borrow the useful ideas from Inferact/agentx-tracking: artifact-backed benchmark rows, best-observed comparisons, filterable serving configuration, an interactive Pareto viewer, and a signed-cookie password gate. The benchmark snapshot itself remains static and requires no database.
The default chart maximizes both axes:
x = 1000 / p90 ITL_ms [tok/user/s]
y = output-token throughput per GPU [tok/GPU/s]
The throughput selector can optionally show aggregate output throughput
(tok/s) without changing the underlying measurements.
The run-type filter separates dummy-weight screening points from real-weight runs. This is independent from validity: a real-weight run can still be a diagnostic rather than an accepted official submission.
A point is on the frontier when no other accepted point is at least as high on both axes and strictly higher on at least one. Changing filters recomputes the frontier. Diagnostic or invalid measurements can be displayed, but they are never allowed onto the accepted frontier.
The “best per concurrency” table is intentionally more specific: it selects the highest-throughput accepted row at each concurrency, using interactivity only to break an exact throughput tie. A multi-objective frontier can contain more than one good answer, so the table should not be confused with the full Pareto set.
Install the Vercel CLI and start the local Vercel runtime:
npm install
npm test
npx vercel devOpen the local URL printed by Vercel. The default shared password is jeffma;
set SITE_PASSWORD and AUTH_SECRET in Vercel for deployed environments.
The import script converts the Vigil Kimi-K3 result schema into the public, browser-ready dataset. It intentionally drops node names and remote artifact paths.
npm run import:data -- --screen /path/to/results.csv
npm testFor another benchmark schema, normalize each record to the fields described in
data/README.md. The UI reads data/sweeps.json; it does not
hard-code particular lane names or concurrencies.
- The TP/TEP × KV data is a noncanonical trend screen using dummy weights, 1,800-second profiling, and the minimum supported AgentX cache warmup.
- The kernel A/B points contain raw metrics but are submission-invalid because profiling metric coverage was below AgentX's required 98% of the configured 3,600 seconds. They are marked diagnostic and excluded from every frontier.
- The 64 GiB “vLLM native offload — 64 GiB control” lanes are native KV-offload controls, not Mooncake results. The tested ROCm environment did not have a working Mooncake runtime.
- A finalist should be rerun with real weights, ten cache-warmup requests per lane, and a 3,600-second profile before making a canonical claim.
The project serves public CSS/JavaScript from public/, while /dashboard and
/api/data require a signed HTTP-only session cookie. Deploy from the linked
Inferact project with:
npx vercel --prod --scope inferact-incConfigure SITE_PASSWORD and a long random AUTH_SECRET in Production and
Preview. Filters are stored in the protected dashboard URL, so the “Copy
shareable view” button recreates the current comparison after sign-in.
MIT