From 02360288a8cce599c80cd51646143e606e44211b Mon Sep 17 00:00:00 2001 From: Bitcoin Universe Date: Sun, 30 Aug 2026 07:39:49 +0000 Subject: [PATCH 01/15] Write down how a source entry reaches an authority that speaks TLS The overlay published index-doge-tap as unreachable from the day it was configured, and the hourly production smoke failed on it for a day, for two reasons this file did not warn about. The entry named http://127.0.0.1:3013 against a service that has served TLS since its certificate was issued. Every probe got an empty reply, which the overlay can only report as unreachable, and the service was running and ready the whole time. The entry now names the certificate's own SAN host over loopback, and the certificate is trusted rather than verification being switched off. The entry also carried the reader's token. The marketplace endpoints check a per-authority token instead, so every checkpoint request was a 401 while the unauthenticated readiness probe passed. A green readiness probe says nothing about the request that actually matters. Both are properties of the deployment rather than of the repository, which is why they were invisible here. The configuration table also gains the certificate file the overlay now reads. Co-Authored-By: Claude Opus 5 --- docs/operations/DEPLOYMENT.md | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/docs/operations/DEPLOYMENT.md b/docs/operations/DEPLOYMENT.md index 16b7d2613c..a6efe7eaae 100644 --- a/docs/operations/DEPLOYMENT.md +++ b/docs/operations/DEPLOYMENT.md @@ -100,6 +100,28 @@ The overlay reads only Universe protocol authorities, configured through bearer tokens supplied by separate named variables. Credentials never appear in the JSON, in responses, or in logs. +Two things about a source entry are easy to get wrong, and both have already +produced an authority the overlay reported as unreachable while it was running +and ready. + +The scheme has to be the one the authority actually speaks. Most authorities +here are plain HTTP on loopback; `index-doge-tap` is an HTTPS listener, so its +entry names `https://doge-tap.internal.bitcoinuniverse.io:3013`. That host is in +the certificate's subject alternative names and resolves to `127.0.0.1` through +`/etc/hosts`, so verification passes and the traffic still stays on the loopback +interface rather than going out to the public address the certificate also +names. The certificate is trusted through `NODE_EXTRA_CA_CERTS`, never by +turning verification off. Point an entry at `http://` when the authority serves +TLS and every probe gets an empty reply, which the overlay can only report as +unreachable. + +The token has to be the one the endpoint being called checks. A service can +carry more than one: `index-doge-tap` authenticates its reader API with +`TOKEN_EXPLORER_BEARER_TOKEN` and its marketplace authority endpoints per +authority, against `DOGE_MARKETPLACE__AUTHORITY_BEARER_TOKEN`. The +readiness probe passing proves nothing about the checkpoint request, because +readiness is unauthenticated. + ## Configuration `/etc/universe-explorer/`, owned by root, group `universe-explorer`, mode 0640: @@ -111,6 +133,7 @@ the JSON, in responses, or in logs. | `gateway.env` | gateway ports and the static root | | `mysql.env` | explorer database name, user, and passwords | | `fulcrum.conf` | the Electrum index this deployment reads | +| `doge-tap-ca.pem` | the certificate `index-doge-tap` presents, trusted by the overlay | ## Build From 5606499da3fa08b7c0da124abc372c897a71eaea Mon Sep 17 00:00:00 2001 From: Bitcoin Universe Date: Sun, 30 Aug 2026 13:25:17 +0000 Subject: [PATCH 02/15] Give Dogecoin and Zcash the full explorer surface Bitcoin has The two chains rendered a capability report where Bitcoin renders a dashboard, and their navigation hid Mining, Charts, and chain Docs behind an activeChain gate. This builds the missing product: - The route registry gains mining, graphs, and docs categories, chain switching preserves the section, and a chart child survives a switch when the destination chain has an equivalent (a registry maps block-sizes-weights to block-sizes and block-health to block-interval). - The chain dashboard: a full-width block timeline with one cube per upcoming target slot and one per confirmed block, heights above every cube, the direction toggle shared with the Bitcoin strip, fee guidance in chain-native units, a Universe Lens over the real pending set (the Bitcoin treemap renderer fed chain-read items, with a table fallback), mining facts, recent blocks with evidence-based miner attribution, and the old capability report filed as a status drawer. - Zcash slots state eligibility, never order: tier chips stay on the cubes and the strip says selection is randomized under ZIP-317. - A chain mining page: network facts, rewards, pool shares with an honest Unknown bucket and named evidence, recent blocks, and a how-this-chain-mines explainer covering AuxPoW and Equihash. - Chain charts: an echarts shell with ten chart routes per chain, ranges disabled until stored history covers them, table fallback, and a pool ranking page. - Chain docs at /dogecoin/docs and /zcash/docs: fourteen sections from dashboard field definitions to the API list, privacy limits on Zcash, and the candidate-versus-projected distinction. - The mempool page keeps its bucket row; the shared cube grammar and page skeleton move to partials so the surfaces cannot drift apart. - Gates: parity-regressions.test.mjs pins the structure in CI, the production smoke checks the new data families with a backfilling-aware honesty rule, the browser smoke audits the rendered timeline, and the visual matrix covers the eight new routes in both themes with fixtures for every new endpoint. Local verification: production build green, 407 universe unit tests, 40-screenshot visual run with zero contrast, a11y, overflow, and console findings. Co-Authored-By: Claude Fable 5 --- .github/workflows/universe-ci.yml | 7 + docs/operations/DEPLOYMENT.md | 18 + .../master-page/master-page.component.html | 13 +- .../master-page/master-page.component.ts | 4 +- frontend/src/app/universe/_chain-cubes.scss | 108 ++++ frontend/src/app/universe/_chain-page.scss | 253 +++++++++ .../chain-dashboard/chain-dashboard-format.ts | 104 ++++ .../chain-dashboard.component.html | 334 +++++++++++ .../chain-dashboard.component.scss | 233 ++++++++ .../chain-dashboard.component.ts | 328 +++++++++++ .../chain-dashboard.service.ts | 122 ++++ .../chain-dashboard/chain-lens.component.html | 77 +++ .../chain-dashboard/chain-lens.component.scss | 84 +++ .../chain-dashboard/chain-lens.component.ts | 207 +++++++ .../chain-dashboard/chain-lens.spec.ts | 102 ++++ .../universe/chain-dashboard/chain-lens.ts | 153 +++++ .../chain-mining.component.html | 213 +++++++ .../chain-mining.component.scss | 147 +++++ .../chain-dashboard/chain-mining.component.ts | 216 +++++++ .../chain-timeline.component.html | 101 ++++ .../chain-timeline.component.scss | 175 ++++++ .../chain-timeline.component.ts | 81 +++ .../chain-dashboard/chain-timeline.spec.ts | 175 ++++++ .../chain-dashboard/chain-timeline.ts | 240 ++++++++ .../universe/chain-docs/chain-docs-content.ts | 98 ++++ .../chain-docs/chain-docs.component.html | 532 ++++++++++++++++++ .../chain-docs/chain-docs.component.scss | 177 ++++++ .../chain-docs/chain-docs.component.spec.ts | 102 ++++ .../chain-docs/chain-docs.component.ts | 102 ++++ .../universe/chain-docs/chain-docs.module.ts | 19 + .../chain-graphs/chain-chart-config.spec.ts | 122 ++++ .../chain-graphs/chain-chart-config.ts | 323 +++++++++++ .../chain-graphs/chain-chart-format.spec.ts | 104 ++++ .../chain-graphs/chain-chart-format.ts | 186 ++++++ .../chain-graphs/chain-chart-options.ts | 222 ++++++++ .../chain-graphs/chain-chart.component.html | 70 +++ .../chain-graphs/chain-chart.component.scss | 148 +++++ .../chain-graphs/chain-chart.component.ts | 399 +++++++++++++ .../chain-graphs/chain-graphs.component.html | 27 + .../chain-graphs/chain-graphs.component.scss | 84 +++ .../chain-graphs/chain-graphs.component.ts | 105 ++++ .../chain-graphs/chain-graphs.module.ts | 48 ++ .../chain-pools-ranking.component.html | 76 +++ .../chain-pools-ranking.component.scss | 147 +++++ .../chain-pools-ranking.component.ts | 163 ++++++ .../multichain-explorer.component.html | 132 +---- .../multichain-explorer.component.scss | 354 +----------- .../multichain-explorer.component.ts | 11 +- .../multichain-explorer.module.ts | 30 +- .../src/app/universe/universe-api.service.ts | 44 ++ .../universe/universe-chain-routing.spec.ts | 72 +++ .../app/universe/universe-chain-routing.ts | 62 +- .../app/universe/universe-chart-registry.ts | 94 ++++ frontend/src/app/universe/universe.types.ts | 190 +++++++ scripts/universe/parity-regressions.test.mjs | 128 +++++ scripts/universe/synthetic-check.mjs | 96 ++++ scripts/universe/visual-qa/capture.mjs | 16 +- scripts/universe/visual-qa/chain-fixtures.mjs | 315 ++++++++++- .../universe/visual-qa/chain-page-audit.mjs | 160 ++++++ .../visual-qa/chain-page-audit.test.mjs | 177 +++++- .../universe/visual-qa/chain-page-smoke.mjs | 130 ++++- scripts/universe/visual-qa/mobile-check.mjs | 3 + 62 files changed, 8250 insertions(+), 513 deletions(-) create mode 100644 frontend/src/app/universe/_chain-cubes.scss create mode 100644 frontend/src/app/universe/_chain-page.scss create mode 100644 frontend/src/app/universe/chain-dashboard/chain-dashboard-format.ts create mode 100644 frontend/src/app/universe/chain-dashboard/chain-dashboard.component.html create mode 100644 frontend/src/app/universe/chain-dashboard/chain-dashboard.component.scss create mode 100644 frontend/src/app/universe/chain-dashboard/chain-dashboard.component.ts create mode 100644 frontend/src/app/universe/chain-dashboard/chain-dashboard.service.ts create mode 100644 frontend/src/app/universe/chain-dashboard/chain-lens.component.html create mode 100644 frontend/src/app/universe/chain-dashboard/chain-lens.component.scss create mode 100644 frontend/src/app/universe/chain-dashboard/chain-lens.component.ts create mode 100644 frontend/src/app/universe/chain-dashboard/chain-lens.spec.ts create mode 100644 frontend/src/app/universe/chain-dashboard/chain-lens.ts create mode 100644 frontend/src/app/universe/chain-dashboard/chain-mining.component.html create mode 100644 frontend/src/app/universe/chain-dashboard/chain-mining.component.scss create mode 100644 frontend/src/app/universe/chain-dashboard/chain-mining.component.ts create mode 100644 frontend/src/app/universe/chain-dashboard/chain-timeline.component.html create mode 100644 frontend/src/app/universe/chain-dashboard/chain-timeline.component.scss create mode 100644 frontend/src/app/universe/chain-dashboard/chain-timeline.component.ts create mode 100644 frontend/src/app/universe/chain-dashboard/chain-timeline.spec.ts create mode 100644 frontend/src/app/universe/chain-dashboard/chain-timeline.ts create mode 100644 frontend/src/app/universe/chain-docs/chain-docs-content.ts create mode 100644 frontend/src/app/universe/chain-docs/chain-docs.component.html create mode 100644 frontend/src/app/universe/chain-docs/chain-docs.component.scss create mode 100644 frontend/src/app/universe/chain-docs/chain-docs.component.spec.ts create mode 100644 frontend/src/app/universe/chain-docs/chain-docs.component.ts create mode 100644 frontend/src/app/universe/chain-docs/chain-docs.module.ts create mode 100644 frontend/src/app/universe/chain-graphs/chain-chart-config.spec.ts create mode 100644 frontend/src/app/universe/chain-graphs/chain-chart-config.ts create mode 100644 frontend/src/app/universe/chain-graphs/chain-chart-format.spec.ts create mode 100644 frontend/src/app/universe/chain-graphs/chain-chart-format.ts create mode 100644 frontend/src/app/universe/chain-graphs/chain-chart-options.ts create mode 100644 frontend/src/app/universe/chain-graphs/chain-chart.component.html create mode 100644 frontend/src/app/universe/chain-graphs/chain-chart.component.scss create mode 100644 frontend/src/app/universe/chain-graphs/chain-chart.component.ts create mode 100644 frontend/src/app/universe/chain-graphs/chain-graphs.component.html create mode 100644 frontend/src/app/universe/chain-graphs/chain-graphs.component.scss create mode 100644 frontend/src/app/universe/chain-graphs/chain-graphs.component.ts create mode 100644 frontend/src/app/universe/chain-graphs/chain-graphs.module.ts create mode 100644 frontend/src/app/universe/chain-graphs/chain-pools-ranking.component.html create mode 100644 frontend/src/app/universe/chain-graphs/chain-pools-ranking.component.scss create mode 100644 frontend/src/app/universe/chain-graphs/chain-pools-ranking.component.ts create mode 100644 frontend/src/app/universe/universe-chart-registry.ts create mode 100644 scripts/universe/parity-regressions.test.mjs diff --git a/.github/workflows/universe-ci.yml b/.github/workflows/universe-ci.yml index b04727686f..a71293aa03 100644 --- a/.github/workflows/universe-ci.yml +++ b/.github/workflows/universe-ci.yml @@ -185,6 +185,13 @@ jobs: working-directory: scripts/universe/visual-qa run: node --test chain-page-audit.test.mjs + # Structural facts of the multichain parity release: navigation not + # gated back to Bitcoin, the route registry whole, the dashboard + # requesting real data, cubes carrying heights, chain-native units. + - name: The parity regressions gate holds + working-directory: scripts/universe + run: node --test parity-regressions.test.mjs + # A fixed port here once measured another job's build: two runners share # a host, the second gateway crashed on the taken port behind "&", the # probe happily reached the first job's server, and the run died halfway diff --git a/docs/operations/DEPLOYMENT.md b/docs/operations/DEPLOYMENT.md index a6efe7eaae..f970fb48b9 100644 --- a/docs/operations/DEPLOYMENT.md +++ b/docs/operations/DEPLOYMENT.md @@ -122,6 +122,24 @@ authority, against `DOGE_MARKETPLACE__AUTHORITY_BEARER_TOKEN`. The readiness probe passing proves nothing about the checkpoint request, because readiness is unauthenticated. +The dashboard, mining, and chart families for Dogecoin and Zcash read two +node RPCs and one durable history store, all configured in the same +`overlay.env`: + +- `UNIVERSE_DOGECOIN_RPC_URL` with `UNIVERSE_DOGECOIN_RPC_USER` and + `UNIVERSE_DOGECOIN_RPC_PASSWORD` (already present for the mempool + collector) also feed the block collector, `estimatesmartfee`, and + `getmininginfo`. +- `UNIVERSE_ZCASH_RPC_URL` points at the Zebra JSON-RPC listener, plain + HTTP on loopback (`http://127.0.0.1:8232/`). Zebra runs without RPC + auth; set `UNIVERSE_ZCASH_RPC_AUTHORIZATION` only if that changes. +- `UNIVERSE_EXPLORER_HISTORY_PATH` names the SQLite file holding collected + block history and mempool samples + (`/var/lib/universe-explorer/chain-history.sqlite`). It is separate from + `UNIVERSE_EXPLORER_STATE_PATH` so history writes never contend with the + mempool collector's commits. Unset, the history features report + themselves unavailable and every other surface keeps working. + ## Configuration `/etc/universe-explorer/`, owned by root, group `universe-explorer`, mode 0640: diff --git a/frontend/src/app/components/master-page/master-page.component.html b/frontend/src/app/components/master-page/master-page.component.html index 8051b04635..8ca085e4ea 100644 --- a/frontend/src/app/components/master-page/master-page.component.html +++ b/frontend/src/app/components/master-page/master-page.component.html @@ -85,8 +85,8 @@ Dashboard - - + - -
-

How much history is readable now

-
    -
  • - {{ reading.label }} - {{ reading.stateLabel }} - {{ reading.detail }} -
    • -
-

Waiting for a block

- - - - - -
-

Questions this chain can answer

-

- Each entry is a claim the indexer makes about itself. Where it says no, the explorer offers no page for that lookup rather than an empty one. -

-
    -
  • - {{ read.label }} - {{ read.stateLabel }} -
    • -
-
- -
-

Protocol indexers

-
    -
  • - {{ protocol.label }} - - {{ protocol.label }} - - {{ protocol.stateLabel }} - - {{ protocol.historyLabel }} - - - {{ protocol.lag.display }} blocks behind - - - - - What it does not cover -
      -
    • {{ reason.text }}
      • -
    -     - - This chain reports it. This explorer has no page for it yet, so there is nothing to open. - -
    • -
-
- -
-

What this chain does differently

-

- Dogecoin fees are quoted per kilobyte, not in sat/vB. This explorer prints the unit the chain actually uses and shows a fee rate only when the first-party source reports one. -

-

- Zcash fee guidance follows ZIP-317 logical actions where the node reports them. A transaction's cost is not a simple function of its byte size on this chain. -

-

- Only the transparent side of Zcash is public. This explorer reports the shape of a shielded transaction, never who was involved or how much moved. -

-

- No projected blocks are shown for this chain. Projecting the next block needs a fee market this explorer does not model here, and a guess presented as a forecast would be worse than nothing. -

-
- - - - -
- Where these readings came from -

- The identifiers behind this page, for checking a reading against the document it was rendered from. -

-
- -
{{ detail.label }}
-
- {{ detail.display }} - - {{ detail.exact }} - - {{ detail.note }} -
- -
-
- - - @@ -1057,7 +929,7 @@

{{ empty.label }}

-

The {{ chainName }} authority answered, and had nothing to report for this request.

diff --git a/frontend/src/app/universe/multichain-explorer/multichain-explorer.component.scss b/frontend/src/app/universe/multichain-explorer/multichain-explorer.component.scss index 6c6c7e9f37..045c22d131 100644 --- a/frontend/src/app/universe/multichain-explorer/multichain-explorer.component.scss +++ b/frontend/src/app/universe/multichain-explorer/multichain-explorer.component.scss @@ -11,6 +11,11 @@ @use '../universe-tokens' as u; +@use '../chain-page' as page; +// The cube grammar shared with the dashboard timeline, so the bucket row +// and the timeline cannot drift apart visually. +@use '../chain-cubes'; + :host { @include u.universe-protocol-tokens; @include u.universe-state-tokens; @@ -18,10 +23,10 @@ display: block; } -.chain-page { - padding: 1.25rem 0 3rem; - color: var(--u-text-primary); -} +// The generic chain-page grammar (head, rail, panels, facts, tables) is the +// shared mixin, so this page and the dashboard, mining, charts, and docs +// pages stay one design. +@include page.universe-chain-page; // --------------------------------------------------------------------------- // Head @@ -32,32 +37,6 @@ // to the end of the rule. Only the chain mark and the save control are added // here, so a Dogecoin transaction page and a Bitcoin one open the same way. -// A chain gets a mark, not a colour that means something. The hue comes from -// the protocol scale, which is held away from every evidence state, and it -// never appears without the chain's name beside it. -.chain-mark { - width: 0.5rem; - height: 0.5rem; - border-radius: var(--u-radius-pill); - flex: none; - align-self: center; - - &.chain-mark-dogecoin { - background: var(--universe-protocol-ordinals); - } - - &.chain-mark-zcash { - background: var(--universe-protocol-alkanes); - } -} - -.lede { - margin: 0 0 var(--u-space-4); - max-width: var(--u-width-prose); - color: var(--u-text-secondary); - line-height: var(--u-leading-normal); -} - .save { flex: none; background: var(--u-surface-raised); @@ -86,99 +65,6 @@ // Status rail // --------------------------------------------------------------------------- -.status-rail { - @include u.universe-surface; - - display: grid; - grid-template-columns: repeat(auto-fit, minmax(9.5rem, 1fr)); - gap: var(--u-space-3) var(--u-space-5); - padding: var(--u-space-3) var(--u-space-4); - margin-bottom: var(--u-space-3); -} - -.reading { - display: flex; - flex-direction: column; - gap: 0.3rem; - min-width: 0; -} - -.reading-label { - color: var(--u-text-muted); - font-size: var(--u-text-2xs); - font-weight: var(--u-weight-semibold); - letter-spacing: var(--u-eyebrow-tracking); - text-transform: uppercase; -} - -.universe-chip { - @include u.universe-chip; -} - -// --------------------------------------------------------------------------- -// Panels -// --------------------------------------------------------------------------- - -.panel { - @include u.universe-surface; - - padding: var(--u-space-4); - margin-bottom: var(--u-space-3); - - h2 { - font-size: var(--u-text-lg); - font-weight: var(--u-weight-semibold); - margin: 0 0 var(--u-space-3); - } - - h3 { - font-size: var(--u-text-base); - font-weight: var(--u-weight-semibold); - color: var(--u-text-secondary); - margin: var(--u-space-4) 0 var(--u-space-2); - } - - h3:first-of-type { - margin-top: 0; - } -} - -.panel-lede { - color: var(--u-text-secondary); - font-size: var(--u-text-sm); - line-height: var(--u-leading-normal); - max-width: var(--u-width-prose); - margin: 0 0 var(--u-space-3); -} - -.notice { - @include u.universe-surface; - - padding: var(--u-space-3) var(--u-space-4); - margin-bottom: var(--u-space-3); - font-size: var(--u-text-sm); - line-height: var(--u-leading-normal); - max-width: var(--u-width-prose); - - &.switched { - border-left: 3px solid var(--universe-state-pending); - } - - &.failure { - border-left: 3px solid var(--universe-state-unavailable); - color: var(--universe-state-unavailable); - } - - &.empty { - border-left: 3px solid var(--universe-state-neutral); - color: var(--u-text-secondary); - } -} - -.secondary { - color: var(--u-text-muted); -} - // --------------------------------------------------------------------------- // Why a chain is not ready // --------------------------------------------------------------------------- @@ -308,53 +194,6 @@ } } -.identifier { - @include u.universe-identifier; -} - -.numeric { - text-align: right; - font-variant-numeric: tabular-nums; - white-space: nowrap; -} - -.dot { - display: inline-block; - width: 3px; - height: 3px; - margin: 0 0.45rem; - border-radius: var(--u-radius-pill); - background: var(--u-text-faint); - vertical-align: middle; -} - -// --------------------------------------------------------------------------- -// Facts -// --------------------------------------------------------------------------- - -.facts { - display: grid; - grid-template-columns: repeat(auto-fit, minmax(13rem, 1fr)); - gap: var(--u-space-3) var(--u-space-5); - margin: 0; - - > div { - min-width: 0; - } - - dt { - color: var(--u-text-muted); - font-size: var(--u-text-xs); - font-weight: var(--u-weight-medium); - } - - dd { - margin: 0.2rem 0 0; - min-width: 0; - overflow-wrap: anywhere; - } -} - .verdict { margin: 0 0 var(--u-space-3); } @@ -615,42 +454,6 @@ // Figures // --------------------------------------------------------------------------- -.figure-row { - display: grid; - grid-template-columns: repeat(auto-fit, minmax(11rem, 1fr)); - gap: var(--u-space-4); -} - -.figure { - display: flex; - flex-direction: column; - gap: 0.2rem; - min-width: 0; -} - -.figure-label { - color: var(--u-text-muted); - font-size: var(--u-text-2xs); - font-weight: var(--u-weight-semibold); - letter-spacing: var(--u-eyebrow-tracking); - text-transform: uppercase; -} - -.figure-value { - font-family: var(--u-font-data); - font-size: var(--u-text-xl); - font-variant-numeric: tabular-nums; - font-weight: var(--u-weight-semibold); - line-height: var(--u-leading-tight); - overflow-wrap: anywhere; -} - -.figure-unit { - color: var(--u-text-muted); - font-size: var(--u-text-sm); - font-weight: var(--u-weight-normal); -} - // The unit belongs to the column, so it is stated once in the header rather // than repeated down every row of a dense table. .column-unit { @@ -730,149 +533,10 @@ } } -// A table wide enough to scroll is a scrollable region, and a region a mouse -// can pan has to be reachable from a keyboard too. It takes focus, so it also -// has to show that it has it. -.table-wrap { - overflow-x: auto; - - &:focus-visible { - outline: 2px solid var(--u-focus-ring); - outline-offset: 2px; - } -} - -table { - width: 100%; - border-collapse: collapse; - font-size: var(--u-text-sm); -} - -thead th { - color: var(--u-text-muted); - font-size: var(--u-text-2xs); - font-weight: var(--u-weight-semibold); - letter-spacing: var(--u-eyebrow-tracking); - text-transform: uppercase; - text-align: left; - padding: 0 var(--u-space-3) var(--u-space-2) 0; - border-bottom: 1px solid var(--u-border); - white-space: nowrap; - - &.numeric { - text-align: right; - } -} - -tbody td { - padding: var(--u-space-2) var(--u-space-3) var(--u-space-2) 0; - border-bottom: 1px solid var(--u-divider); - vertical-align: top; - overflow-wrap: anywhere; -} - -tbody tr:last-child td { - border-bottom: 0; -} - // --------------------------------------------------------------------------- // Candidate buckets // --------------------------------------------------------------------------- -// The same cube grammar the Bitcoin projection uses: a face carrying the -// figures, a skewed top and side for depth, the projected-block palette. -// What differs is the claim, which the copy above the row states. -.cube-row { - display: flex; - gap: calc(var(--u-space-6) + 20px); - overflow-x: auto; - padding: calc(0.24 * var(--cube-size)) var(--u-space-2) var(--u-space-2) - calc(0.16 * var(--cube-size) + var(--u-space-2)); - --cube-size: 125px; - - &:focus-visible { - outline: 2px solid var(--u-focus-ring); - outline-offset: 2px; - } -} - -.candidate-cube { - flex: 0 0 auto; - width: var(--cube-size); -} - -.cube-block { - position: relative; - width: var(--cube-size); - height: var(--cube-size); - background: var(--u-block-projected-empty); -} - -.cube-block::after { - content: ''; - width: var(--cube-size); - height: calc(0.192 * var(--cube-size)); - position: absolute; - top: calc(-0.192 * var(--cube-size)); - left: calc(-0.16 * var(--cube-size)); - background-color: var(--u-block-projected-top); - transform: skew(40deg); - transform-origin: top; -} - -.cube-block::before { - content: ''; - width: calc(0.16 * var(--cube-size)); - height: var(--cube-size); - position: absolute; - top: calc(-0.096 * var(--cube-size)); - left: calc(-0.16 * var(--cube-size)); - background-color: var(--u-block-projected-side); - transform: skewY(50deg); - transform-origin: top; -} - -.cube-body { - position: relative; - height: 100%; - display: flex; - flex-direction: column; - align-items: center; - justify-content: center; - gap: var(--u-space-1); - text-align: center; - padding: var(--u-space-2); - color: var(--u-block-ink); -} - -.cube-median { - font-size: var(--u-text-md); - font-weight: var(--u-weight-bold); - overflow-wrap: anywhere; -} - -.cube-unit { - font-size: var(--u-text-2xs); - font-weight: var(--u-weight-regular); -} - -.cube-span, -.cube-count, -.cube-size { - font-size: var(--u-text-2xs); - color: var(--u-block-ink-muted); - overflow-wrap: anywhere; -} - -.cube-caption { - margin-top: var(--u-space-2); - display: flex; - flex-direction: column; - align-items: flex-start; - gap: var(--u-space-1); - font-size: var(--u-text-2xs); -} - .bucket-facts { margin-top: var(--u-space-3); } diff --git a/frontend/src/app/universe/multichain-explorer/multichain-explorer.component.ts b/frontend/src/app/universe/multichain-explorer/multichain-explorer.component.ts index 0fc63cc67a..c6fd6e069e 100644 --- a/frontend/src/app/universe/multichain-explorer/multichain-explorer.component.ts +++ b/frontend/src/app/universe/multichain-explorer/multichain-explorer.component.ts @@ -98,7 +98,6 @@ import { } from 'rxjs'; type MultichainPage = - | 'dashboard' | 'mempool' | 'block' | 'transaction' @@ -225,7 +224,7 @@ export class MultichainExplorerComponent implements OnInit, OnDestroy { readonly chainName: string; readonly ticker: string; readonly protocolIds: readonly string[]; - page: MultichainPage = 'dashboard'; + page: MultichainPage = 'mempool'; reference = ''; saved = false; switchedFrom: string | null = null; @@ -436,8 +435,6 @@ export class MultichainExplorerComponent implements OnInit, OnDestroy { pageLabel(): string { switch (this.page) { - case 'dashboard': - return $localize`:@@universe.chain.page-dashboard:overview`; case 'mempool': return $localize`:@@universe.chain.page-mempool:pending transactions`; case 'block': @@ -466,8 +463,6 @@ export class MultichainExplorerComponent implements OnInit, OnDestroy { /** One sentence saying what the reader is looking at, under the heading. */ pageLede(): string { switch (this.page) { - case 'dashboard': - return $localize`:@@universe.chain.lede-dashboard:What this explorer can answer about ${this.chainName}:CHAIN: right now, and how far behind the chain tip each answer is.`; case 'mempool': return $localize`:@@universe.chain.lede-mempool:Transactions seen by our own ${this.chainName}:CHAIN: node and not yet in a block.`; case 'protocols': @@ -629,8 +624,6 @@ export class MultichainExplorerComponent implements OnInit, OnDestroy { const reference = context.params.get('reference') ?? ''; const protocol = context.params.get('protocol') ?? ''; switch (context.page) { - case 'dashboard': - return of(null); case 'mempool': return this.api.getChainMempool$(this.chain); case 'transaction': @@ -744,8 +737,6 @@ export class MultichainExplorerComponent implements OnInit, OnDestroy { private pageDescription(context: RequestContext): string { const reference = this.referenceFrom(context); switch (context.page) { - case 'dashboard': - return $localize`:@@universe.chain.meta-dashboard:What Universe Explorer can answer about ${this.chainName}:CHAIN: right now, how far behind the chain tip each answer is, and which protocol indexers are running.`; case 'mempool': return $localize`:@@universe.chain.meta-mempool:${this.chainName}:CHAIN: transactions seen by Bitcoin Universe's own node and not yet in a block, read from first-party data.`; case 'protocols': diff --git a/frontend/src/app/universe/multichain-explorer/multichain-explorer.module.ts b/frontend/src/app/universe/multichain-explorer/multichain-explorer.module.ts index 4c0fb0a89a..275d9dedd4 100644 --- a/frontend/src/app/universe/multichain-explorer/multichain-explorer.module.ts +++ b/frontend/src/app/universe/multichain-explorer/multichain-explorer.module.ts @@ -1,13 +1,34 @@ import { NgModule } from '@angular/core'; import { RouterModule, Routes } from '@angular/router'; import { MultichainExplorerComponent } from '@app/universe/multichain-explorer/multichain-explorer.component'; +import { ChainDashboardComponent } from '@app/universe/chain-dashboard/chain-dashboard.component'; +import { ChainMiningComponent } from '@app/universe/chain-dashboard/chain-mining.component'; const routes: Routes = [ { path: '', - component: MultichainExplorerComponent, + component: ChainDashboardComponent, data: { page: 'dashboard' }, }, + { + path: 'mining', + component: ChainMiningComponent, + data: { page: 'mining' }, + }, + { + path: 'graphs', + loadChildren: () => + import('@app/universe/chain-graphs/chain-graphs.module').then( + (m) => m.ChainGraphsModule + ), + }, + { + path: 'docs', + loadChildren: () => + import('@app/universe/chain-docs/chain-docs.module').then( + (m) => m.ChainDocsModule + ), + }, { path: 'mempool', component: MultichainExplorerComponent, @@ -61,6 +82,11 @@ const routes: Routes = [ ]; @NgModule({ - imports: [MultichainExplorerComponent, RouterModule.forChild(routes)], + imports: [ + MultichainExplorerComponent, + ChainDashboardComponent, + ChainMiningComponent, + RouterModule.forChild(routes), + ], }) export class MultichainExplorerModule {} diff --git a/frontend/src/app/universe/universe-api.service.ts b/frontend/src/app/universe/universe-api.service.ts index 5f1c16540b..40282a55b7 100644 --- a/frontend/src/app/universe/universe-api.service.ts +++ b/frontend/src/app/universe/universe-api.service.ts @@ -17,8 +17,14 @@ import { OrdRuneView, OrdSatView, ChainCapabilityEnvelope, + ChainDashboardView, ChainExplorerPayload, + ChartSeriesView, ExplorerChain, + FeeRecommendationsView, + MiningPoolsView, + MiningSummaryView, + RecentBlocksView, UniverseSearchResponse, } from '@app/universe/universe.types'; @@ -170,6 +176,44 @@ export class UniverseApiService { ); } + /** The one-call dashboard aggregate: blocks, buckets, fees, mempool, mining. */ + getChainDashboard$(chain: Exclude): Observable { + return this.httpClient.get( + this.apiBaseUrl + '/api/v1/' + chain + '/dashboard?network=mainnet' + ); + } + + getChainRecentBlocks$(chain: Exclude, limit = 15): Observable { + return this.httpClient.get( + this.apiBaseUrl + '/api/v1/' + chain + '/blocks/recent?network=mainnet&limit=' + limit + ); + } + + getChainFees$(chain: Exclude): Observable { + return this.httpClient.get( + this.apiBaseUrl + '/api/v1/' + chain + '/fees?network=mainnet' + ); + } + + getChainMining$(chain: Exclude): Observable { + return this.httpClient.get( + this.apiBaseUrl + '/api/v1/' + chain + '/mining?network=mainnet' + ); + } + + getChainMiningPools$(chain: Exclude, window = '1w'): Observable { + return this.httpClient.get( + this.apiBaseUrl + '/api/v1/' + chain + '/mining/pools?network=mainnet&window=' + encodeURIComponent(window) + ); + } + + getChainChartSeries$(chain: Exclude, seriesId: string, range = '1w'): Observable { + return this.httpClient.get( + this.apiBaseUrl + '/api/v1/' + chain + '/charts/' + encodeURIComponent(seriesId) + + '?network=mainnet&range=' + encodeURIComponent(range) + ); + } + getChainTransaction$(chain: Exclude, txid: string): Observable { return this.httpClient.get( this.apiBaseUrl + '/api/v1/' + chain + '/tx/' + encodeURIComponent(txid) + '?network=mainnet' diff --git a/frontend/src/app/universe/universe-chain-routing.spec.ts b/frontend/src/app/universe/universe-chain-routing.spec.ts index dcff36fe66..8f4903f167 100644 --- a/frontend/src/app/universe/universe-chain-routing.spec.ts +++ b/frontend/src/app/universe/universe-chain-routing.spec.ts @@ -1,4 +1,5 @@ import { + ExplorerRouteCategory, explorerChainFromUrl, explorerRouteCategory, explorerSectionRoute, @@ -33,4 +34,75 @@ describe('Universe chain routing', () => { droppedObject: true, }); }); + + // The route category type is the registry every chain's navigation is built + // from. Losing a section here silently removes it from every chain's menu, + // so the full set is pinned. + it('keeps every product section in the category registry', () => { + const sections: ExplorerRouteCategory[] = [ + 'dashboard', 'mining', 'mempool', 'protocols', 'graphs', 'docs', + ]; + for (const section of sections) { + expect(explorerSectionRoute('bitcoin', section as Exclude)).toBeTruthy(); + expect(explorerSectionRoute('dogecoin', section as Exclude)).toContain('/dogecoin'); + expect(explorerSectionRoute('zcash', section as Exclude)).toContain('/zcash'); + } + }); + + it('classifies mining, graphs, and docs URLs as sections, not objects', () => { + expect(explorerRouteCategory('/mining')).toBe('mining'); + expect(explorerRouteCategory('/dogecoin/mining')).toBe('mining'); + expect(explorerRouteCategory('/zcash/graphs')).toBe('graphs'); + expect(explorerRouteCategory('/graphs/mining/pools')).toBe('graphs'); + expect(explorerRouteCategory('/dogecoin/docs')).toBe('docs'); + expect(explorerRouteCategory('/docs/api')).toBe('docs'); + // A pool page names one pool, which another chain does not have. + expect(explorerRouteCategory('/mining/pool/foundry')).toBe('object'); + }); + + it('preserves the section when switching chains', () => { + expect(explorerSwitchTarget('/mining', 'dogecoin')).toEqual({ + path: '/dogecoin/mining', + droppedObject: false, + }); + expect(explorerSwitchTarget('/dogecoin/mining', 'zcash')).toEqual({ + path: '/zcash/mining', + droppedObject: false, + }); + expect(explorerSwitchTarget('/zcash/docs', 'bitcoin')).toEqual({ + path: '/docs', + droppedObject: false, + }); + }); + + it('preserves a chart child route across chains when an equivalent exists', () => { + expect(explorerSwitchTarget('/graphs/mining/pools', 'dogecoin')).toEqual({ + path: '/dogecoin/graphs/mining/pools', + droppedObject: false, + }); + expect( + explorerSwitchTarget('/dogecoin/graphs/mining/hashrate-difficulty', 'zcash') + ).toEqual({ + path: '/zcash/graphs/mining/hashrate-difficulty', + droppedObject: false, + }); + // Sizes and weights exist together only on Bitcoin; the others chart sizes. + expect( + explorerSwitchTarget('/graphs/mining/block-sizes-weights', 'zcash') + ).toEqual({ + path: '/zcash/graphs/mining/block-sizes', + droppedObject: false, + }); + expect( + explorerSwitchTarget('/dogecoin/graphs/mining/block-sizes', 'bitcoin') + ).toEqual({ + path: '/graphs/mining/block-sizes-weights', + droppedObject: false, + }); + // The fiat price chart has no self-hosted equivalent on the other chains. + expect(explorerSwitchTarget('/graphs/price', 'dogecoin')).toEqual({ + path: '/dogecoin/graphs', + droppedObject: false, + }); + }); }); diff --git a/frontend/src/app/universe/universe-chain-routing.ts b/frontend/src/app/universe/universe-chain-routing.ts index 4d5ae2d485..df13dfec41 100644 --- a/frontend/src/app/universe/universe-chain-routing.ts +++ b/frontend/src/app/universe/universe-chain-routing.ts @@ -1,7 +1,8 @@ import { ExplorerChain } from '@app/universe/universe.types'; +import { equivalentGraphChild } from '@app/universe/universe-chart-registry'; export type ExplorerRouteCategory = - 'dashboard' | 'mempool' | 'protocols' | 'object'; + 'dashboard' | 'mining' | 'mempool' | 'protocols' | 'graphs' | 'docs' | 'object'; /** * How each chain is written when a person reads it. The wire uses lowercase @@ -40,6 +41,19 @@ export function explorerRouteCategory(url: string): ExplorerRouteCategory { if (segments.length === 1 && segments[0] === 'protocols') { return 'protocols'; } + // The mining dashboard is a section; a pool page under it addresses one + // pool, which does not exist on another chain, so it stays an object. + if (segments.length === 1 && segments[0] === 'mining') { + return 'mining'; + } + // Every chart child stays inside the graphs section: the switcher decides + // per chart whether the destination chain has an equivalent. + if (segments[0] === 'graphs') { + return 'graphs'; + } + if (segments[0] === 'docs') { + return 'docs'; + } return 'object'; } @@ -51,26 +65,46 @@ export function explorerSectionRoute( if (category === 'mempool') { return '/pulse'; } - if (category === 'protocols') { - return '/protocols'; - } - return '/'; + return category === 'dashboard' ? '/' : `/${category}`; } return category === 'dashboard' ? `/${chain}` : `/${chain}/${category}`; } +/** The chart child path after `graphs/`, or null when the URL is not one. */ +function graphChildFrom(url: string): string | null { + const segments = url.split(/[?#]/, 1)[0].split('/').filter(Boolean); + if (segments[0] === 'dogecoin' || segments[0] === 'zcash') { + segments.shift(); + } + if (segments[0] !== 'graphs' || segments.length < 2) { + return null; + } + return segments.slice(1).join('/'); +} + export function explorerSwitchTarget( currentUrl: string, targetChain: ExplorerChain ): { readonly path: string; readonly droppedObject: boolean } { const category = explorerRouteCategory(currentUrl); - return category === 'object' - ? { - path: explorerSectionRoute(targetChain, 'dashboard'), - droppedObject: true, - } - : { - path: explorerSectionRoute(targetChain, category), - droppedObject: false, - }; + if (category === 'object') { + return { + path: explorerSectionRoute(targetChain, 'dashboard'), + droppedObject: true, + }; + } + if (category === 'graphs') { + const child = graphChildFrom(currentUrl); + const equivalent = child ? equivalentGraphChild(child, targetChain) : null; + return { + path: equivalent + ? `${explorerSectionRoute(targetChain, 'graphs')}/${equivalent}` + : explorerSectionRoute(targetChain, 'graphs'), + droppedObject: false, + }; + } + return { + path: explorerSectionRoute(targetChain, category), + droppedObject: false, + }; } diff --git a/frontend/src/app/universe/universe-chart-registry.ts b/frontend/src/app/universe/universe-chart-registry.ts new file mode 100644 index 0000000000..a3089639b0 --- /dev/null +++ b/frontend/src/app/universe/universe-chart-registry.ts @@ -0,0 +1,94 @@ +import { ExplorerChain } from '@app/universe/universe.types'; + +/** + * The chart child routes each chain offers under its graphs section. + * + * One table, used by the chain switcher to decide whether a chart survives a + * chain switch and by the chain graphs page to build its navigation. A chart + * missing from a chain's list is not hidden by accident: it either has a + * different spelling there (see {@link GRAPH_ROUTE_EQUIVALENTS}) or no honest + * equivalent exists on that chain. + */ +export const CHAIN_GRAPH_CHILDREN: Record = { + bitcoin: [ + 'mempool', + 'mining/hashrate-difficulty', + 'mining/pools-dominance', + 'mining/pools', + 'mining/block-fees', + 'mining/block-fees-subsidy', + 'mining/block-rewards', + 'mining/block-fee-rates', + 'mining/block-sizes-weights', + 'mining/block-health', + 'price', + ], + dogecoin: [ + 'mempool', + 'mining/hashrate-difficulty', + 'mining/pools-dominance', + 'mining/pools', + 'mining/block-fees', + 'mining/block-fees-subsidy', + 'mining/block-rewards', + 'mining/block-fee-rates', + 'mining/block-sizes', + 'mining/block-interval', + ], + zcash: [ + 'mempool', + 'mining/hashrate-difficulty', + 'mining/pools-dominance', + 'mining/pools', + 'mining/block-fees', + 'mining/block-fees-subsidy', + 'mining/block-rewards', + 'mining/block-fee-rates', + 'mining/block-sizes', + 'mining/block-interval', + ], +}; + +/** + * Where a chart keeps its meaning across chains under a different spelling. + * + * Bitcoin plots sizes and weights together because segwit gave it two + * capacity measures; the other chains have one, so their chart is sizes + * alone. Block health grades Bitcoin blocks against this explorer's own + * projection, which the other chains honestly cannot have yet, so their + * nearest reliability reading is the observed against target interval. + */ +const GRAPH_ROUTE_EQUIVALENTS: Record> = { + 'mining/block-sizes-weights': { + dogecoin: 'mining/block-sizes', + zcash: 'mining/block-sizes', + }, + 'mining/block-sizes': { + bitcoin: 'mining/block-sizes-weights', + }, + 'mining/block-health': { + dogecoin: 'mining/block-interval', + zcash: 'mining/block-interval', + }, + 'mining/block-interval': { + bitcoin: 'mining/block-health', + }, +}; + +/** + * The chart child a switch to `targetChain` should land on, or null when the + * target chain has no equivalent and the switch should land on its graphs + * landing page instead. + */ +export function equivalentGraphChild( + child: string, + targetChain: ExplorerChain +): string | null { + if (CHAIN_GRAPH_CHILDREN[targetChain].includes(child)) { + return child; + } + const mapped = GRAPH_ROUTE_EQUIVALENTS[child]?.[targetChain]; + return mapped && CHAIN_GRAPH_CHILDREN[targetChain].includes(mapped) + ? mapped + : null; +} diff --git a/frontend/src/app/universe/universe.types.ts b/frontend/src/app/universe/universe.types.ts index 56e77e6378..cc3403f12a 100644 --- a/frontend/src/app/universe/universe.types.ts +++ b/frontend/src/app/universe/universe.types.ts @@ -365,6 +365,196 @@ export interface ChainCapabilityEnvelope { export type ChainExplorerPayload = Record; +// --- Chain dashboard, mining, fees, and chart views --- +// Mirrors backend-apis src/universe-explorer/contracts/chain-dashboard.ts. +// Quantities are exact decimal strings; unknown facts are null, never zero. + +export type PoolAttributionEvidence = + | 'coinbase-tag' + | 'payout-address' + | 'auxpow-parent-tag'; + +export interface BlockMinerView { + poolId: string | null; + name: string | null; + evidence: PoolAttributionEvidence | null; +} + +export interface ChainBlockSummary { + heightAtomic: string; + hash: string; + time: string; + txCountAtomic: string; + sizeBytesAtomic: string | null; + feesAtomic: string | null; + subsidyAtomic: string | null; + rewardAtomic: string | null; + medianFeeRateDecimal: string | null; + difficultyDecimal: string | null; + intervalSecondsAtomic: string | null; + miner: BlockMinerView; +} + +export interface RecentBlocksView { + schemaVersion: string; + chain: ExplorerChain; + network: ExplorerNetwork; + tip: SourceCheckpoint | null; + blocks: ChainBlockSummary[]; + coverage: { + fromHeightAtomic: string | null; + toHeightAtomic: string | null; + complete: boolean; + }; + observedAt: string; +} + +export type FeeRecommendationBasis = + | 'node-estimate' + | 'mempool-quantile' + | 'recent-blocks' + | 'relay-floor' + | 'zip317-conventional'; + +export interface FeeRecommendationLevel { + id: string; + amountDecimal: string; + basis: FeeRecommendationBasis; +} + +export type FeeRecommendationsView = + | { + schemaVersion: string; + chain: ExplorerChain; + network: ExplorerNetwork; + kind: 'fee-per-kilobyte'; + unit: 'koinu/kB'; + levels: FeeRecommendationLevel[]; + minRelayFeeAtomicPerKb: string; + tip: SourceCheckpoint | null; + observedAt: string; + } + | { + schemaVersion: string; + chain: ExplorerChain; + network: ExplorerNetwork; + kind: 'zip-317'; + unit: 'zatoshi'; + marginalFeeAtomic: string; + graceActionsAtomic: string; + typicalConventionalFeeAtomic: string; + paidShareDecimal: string | null; + basis: FeeRecommendationBasis; + tip: SourceCheckpoint | null; + observedAt: string; + }; + +export type HashrateUnit = 'hashes-per-second' | 'solutions-per-second'; + +export interface MiningSummaryView { + schemaVersion: string; + chain: ExplorerChain; + network: ExplorerNetwork; + tip: SourceCheckpoint | null; + difficultyDecimal: string | null; + networkRateDecimal: string | null; + hashrateUnit: HashrateUnit; + algorithm: string; + targetBlockSecondsAtomic: string; + observedIntervalSecondsDecimal: string | null; + windowBlocksAtomic: string | null; + subsidyAtomic: string | null; + meanRewardAtomic: string | null; + meanFeesAtomic: string | null; + mergedMining: { + supported: boolean; + noticeId: string | null; + }; + observedAt: string; +} + +export interface MiningPoolShare { + poolId: string; + name: string; + blocksAtomic: string; + shareDecimal: string; + evidence: PoolAttributionEvidence[]; +} + +export interface MiningPoolsView { + schemaVersion: string; + chain: ExplorerChain; + network: ExplorerNetwork; + windowId: string; + windowBlocksAtomic: string; + pools: MiningPoolShare[]; + attributionDatasetVersion: string; + coverageComplete: boolean; + observedAt: string; +} + +export interface ChartSeriesLine { + key: string; + unit: string; + points: [string, string | null][]; +} + +export interface ChartSeriesView { + schemaVersion: string; + chain: ExplorerChain; + network: ExplorerNetwork; + seriesId: string; + rangeId: string; + lines: ChartSeriesLine[]; + aggregation: string; + bucketSecondsAtomic: string | null; + coverage: { + fromAtomic: string | null; + toAtomic: string | null; + complete: boolean; + earliestAtomic: string | null; + }; + sourceHeightAtomic: string | null; + observedAt: string; +} + +export interface ChainMempoolSummary { + txCountAtomic: string | null; + totalSizeBytesAtomic: string | null; + totalFeesAtomic: string | null; + arrivalRatePerSecondDecimal: string | null; + observedAt: string | null; +} + +export type ChainSubsystemId = + | 'core-node' + | 'confirmed-history' + | 'address-history' + | 'mempool' + | 'mining-analytics' + | 'historical-statistics' + | 'protocol-indexers'; + +export interface ChainSubsystemHealth { + id: ChainSubsystemId; + state: 'ready' | 'degraded' | 'unavailable'; + reasonIds: string[]; +} + +export interface ChainDashboardView { + schemaVersion: string; + chain: ExplorerChain; + network: ExplorerNetwork; + tip: SourceCheckpoint | null; + recentBlocks: RecentBlocksView | null; + buckets: ChainExplorerPayload | null; + fees: FeeRecommendationsView | null; + mempool: ChainMempoolSummary | null; + mining: MiningSummaryView | null; + subsystems: ChainSubsystemHealth[]; + observedAt: string; +} + export interface UniverseSearchResult { chain: ExplorerChain; network: 'mainnet'; diff --git a/scripts/universe/parity-regressions.test.mjs b/scripts/universe/parity-regressions.test.mjs new file mode 100644 index 0000000000..508e5c7ba7 --- /dev/null +++ b/scripts/universe/parity-regressions.test.mjs @@ -0,0 +1,128 @@ +/** + * Regression gate for the multichain parity release. + * + * Every case here pins a structural fact that once failed in production or + * that a cleanup could plausibly reintroduce: a navigation item gated back + * to Bitcoin, a section dropped from the route registry, a dashboard whose + * data request went back to answering nothing, cubes losing their heights, + * or Bitcoin units wearing another chain's name. These read the source + * rather than a build, so they run in seconds and fail with the file named. + */ +import assert from 'node:assert/strict'; +import { readFileSync } from 'node:fs'; +import { dirname, join } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import test from 'node:test'; + +const root = join(dirname(fileURLToPath(import.meta.url)), '..', '..'); + +function read(path) { + return readFileSync(join(root, path), 'utf8'); +} + +test('Mining and Charts navigation are not gated to Bitcoin', () => { + const nav = read('frontend/src/app/components/master-page/master-page.component.html'); + const miningItem = nav.match(/
  • /)?.[0] ?? ''; + assert.ok(miningItem.length, 'the mining navigation item exists'); + assert.ok( + !/\*ngIf="activeChain === 'bitcoin'/.test(miningItem), + 'the mining item must not require activeChain to be bitcoin', + ); + assert.match(miningItem, /chainRoute\('mining'\)/, + 'the mining item routes through the chain-aware registry'); + const graphsItem = nav.match(/id="btn-graphs"[\s\S]*?<\/li>/)?.[0] ?? ''; + assert.ok( + !/activeChain === 'bitcoin'"/.test(graphsItem.split('>')[0]), + 'the charts item must not be hidden on other chains', + ); + assert.match(graphsItem, /chainRoute\('graphs'\)/); +}); + +test('Docs navigation stays inside the selected chain', () => { + const nav = read('frontend/src/app/components/master-page/master-page.component.html'); + const docsItem = nav.match(/id="btn-docs"[\s\S]*?<\/li>/)?.[0] ?? ''; + assert.match(docsItem, /chainRoute\('docs'\)/, + 'a Dogecoin or Zcash reader who opens Docs must get chain docs'); +}); + +test('the route category registry keeps every product section', () => { + const routing = read('frontend/src/app/universe/universe-chain-routing.ts'); + for (const section of ['dashboard', 'mining', 'mempool', 'protocols', 'graphs', 'docs']) { + assert.ok( + routing.includes(`'${section}'`), + `route category registry names ${section}`, + ); + } +}); + +test('the multichain route table mounts mining, graphs, and docs', () => { + const module_ = read('frontend/src/app/universe/multichain-explorer/multichain-explorer.module.ts'); + assert.match(module_, /path: 'mining'/); + assert.match(module_, /path: 'graphs'/); + assert.match(module_, /path: 'docs'/); + assert.match(module_, /chain-graphs\.module/); + assert.match(module_, /chain-docs\.module/); +}); + +test('the dashboard requests real data, not of(null)', () => { + const multichain = read('frontend/src/app/universe/multichain-explorer/multichain-explorer.component.ts'); + assert.ok( + !multichain.includes("case 'dashboard':"), + 'the status-report component no longer owns the dashboard', + ); + const dashboard = read('frontend/src/app/universe/chain-dashboard/chain-dashboard.component.ts'); + assert.match(dashboard, /dashboard\$\(/, 'the dashboard subscribes to the data service'); + const service = read('frontend/src/app/universe/chain-dashboard/chain-dashboard.service.ts'); + assert.match(service, /getChainDashboard\$/, 'the service calls the dashboard aggregate'); +}); + +test('the dashboard composes the block timeline, and cubes carry heights', () => { + const dashboard = read('frontend/src/app/universe/chain-dashboard/chain-dashboard.component.html'); + assert.match(dashboard, / { + for (const path of [ + 'frontend/src/app/universe/chain-dashboard/chain-dashboard.component.html', + 'frontend/src/app/universe/chain-dashboard/chain-mining.component.html', + 'frontend/src/app/universe/chain-dashboard/chain-timeline.component.html', + 'frontend/src/app/universe/multichain-explorer/multichain-explorer.component.html', + ]) { + const template = read(path); + assert.ok(!template.includes('sat/vB'), `${path} must not print sat/vB`); + assert.ok(!/\bBTC\b/.test(template), `${path} must not print BTC`); + } +}); + +test('Zcash ordering is never presented as deterministic', () => { + const reader = read('frontend/src/app/universe/chain-dashboard/chain-timeline.ts'); + assert.match(reader, /zip317-eligibility-tiers/, + 'the tier semantics drive a disclosure'); + assert.match(reader, /randomized/i, + 'the disclosure names the randomized selection'); +}); + +test('the visual gate covers the parity routes', () => { + const capture = read('scripts/universe/visual-qa/capture.mjs'); + for (const id of [ + 'dogecoin-mining', 'dogecoin-graphs', 'dogecoin-docs', + 'zcash-mining', 'zcash-graphs', 'zcash-docs', + ]) { + assert.ok(capture.includes(`'${id}'`), `capture gate covers ${id}`); + } +}); + +test('the production smoke checks the dashboard families for both chains', () => { + const smoke = read('scripts/universe/synthetic-check.mjs'); + assert.match(smoke, /checkChainDashboardFamilies/); + assert.match(smoke, /universe-chain-dashboard-v1/); + assert.match(smoke, /universe-recent-blocks-v1/); + assert.match(smoke, /universe-chart-series-v1/); +}); diff --git a/scripts/universe/synthetic-check.mjs b/scripts/universe/synthetic-check.mjs index d0f306362e..7d209ed9f2 100644 --- a/scripts/universe/synthetic-check.mjs +++ b/scripts/universe/synthetic-check.mjs @@ -461,6 +461,97 @@ async function checkChainReadsAreServed(chain, envelope) { */ const COMMIT_SHA = /^[0-9a-f]{7,64}$/; +/** + * The dashboard data families behind the chain dashboard, mining, and + * charts pages. Every route must be mounted and answer its own schema; a + * missing route here is what a stale overlay looks like when the frontend + * already advertises the page. + * + * Empty history is judged against the deployment's own claim: while the + * dashboard reports historical statistics as backfilling, an empty block + * list is honest and noted; once it reports ready, empty becomes a failure, + * because a chart route answering 200 with nothing to draw is the exact + * state this suite exists to catch. + */ +async function checkChainDashboardFamilies(chain) { + const label = `chain:${chain}:dashboard`; + const dashboard = await get(`/api/v1/${chain}/dashboard`); + if (dashboard.status !== 200 || dashboard.body?.schemaVersion !== 'universe-chain-dashboard-v1') { + fail(label, `/api/v1/${chain}/dashboard answered HTTP ${dashboard.status} (${namedReason(dashboard.body)})`); + return; + } + const subsystems = Array.isArray(dashboard.body.subsystems) ? dashboard.body.subsystems : []; + if (!subsystems.length) { + fail(label, 'the dashboard names no subsystems, so nothing scopes a failure'); + } else { + pass(label, `dashboard answers with ${subsystems.length} subsystem readings`); + } + const history = subsystems.find((entry) => entry?.id === 'historical-statistics'); + const historyReady = history?.state === 'ready'; + + const blocks = await get(`/api/v1/${chain}/blocks/recent?limit=12`); + if (blocks.status !== 200 || blocks.body?.schemaVersion !== 'universe-recent-blocks-v1') { + fail(label, `/blocks/recent answered HTTP ${blocks.status} (${namedReason(blocks.body)})`); + } else if (!Array.isArray(blocks.body.blocks) || blocks.body.blocks.length === 0) { + if (historyReady) { + fail(label, 'historical statistics claim ready while the block list is empty'); + } else { + notes.push(`${label}: block history is still backfilling, list empty and said so`); + } + } else { + const newest = blocks.body.blocks[0]; + if (typeof newest.heightAtomic !== 'string' || typeof newest.hash !== 'string') { + fail(label, 'a recent block is missing its height or hash'); + } else { + pass(label, `recent blocks answer, newest at height ${newest.heightAtomic}`); + } + } + + const fees = await get(`/api/v1/${chain}/fees`); + if (fees.status !== 200 || fees.body?.schemaVersion !== 'universe-fee-recommendations-v1') { + fail(label, `/fees answered HTTP ${fees.status} (${namedReason(fees.body)})`); + } else if (fees.body.kind !== 'fee-per-kilobyte' && fees.body.kind !== 'zip-317') { + fail(label, `fee guidance kind is ${JSON.stringify(fees.body.kind)}`); + } else { + pass(label, `fee guidance answers as ${fees.body.kind}`); + } + + const mining = await get(`/api/v1/${chain}/mining`); + if (mining.status !== 200 || mining.body?.schemaVersion !== 'universe-mining-summary-v1') { + fail(label, `/mining answered HTTP ${mining.status} (${namedReason(mining.body)})`); + } else { + pass(label, `mining summary answers (difficulty ${mining.body.difficultyDecimal ?? 'not reported'})`); + } + + const pools = await get(`/api/v1/${chain}/mining/pools?window=1w`); + if (pools.status !== 200 || pools.body?.schemaVersion !== 'universe-mining-pools-v1') { + fail(label, `/mining/pools answered HTTP ${pools.status} (${namedReason(pools.body)})`); + } else if (!Array.isArray(pools.body.pools)) { + fail(label, 'pool shares answered without a pools array'); + } else { + pass(label, `pool shares answer with ${pools.body.pools.length} rows over ${pools.body.windowBlocksAtomic} blocks`); + } + + for (const seriesId of ['block-fees', 'mempool-count']) { + const series = await get(`/api/v1/${chain}/charts/${seriesId}?range=24h`); + if (series.status !== 200 || series.body?.schemaVersion !== 'universe-chart-series-v1') { + fail(label, `/charts/${seriesId} answered HTTP ${series.status} (${namedReason(series.body)})`); + continue; + } + const points = (series.body.lines ?? []).reduce( + (sum, line) => sum + (Array.isArray(line?.points) ? line.points.length : 0), + 0, + ); + if (points === 0 && historyReady) { + fail(label, `/charts/${seriesId} answered 200 with nothing to draw while history claims ready`); + } else if (points === 0) { + notes.push(`${label}: ${seriesId} has no samples yet, and history says it is backfilling`); + } else { + pass(label, `${seriesId} has ${points} drawable points over 24h`); + } + } +} + function checkChainReleaseIdentity(envelopes) { const named = new Map(); for (const [chain, envelope] of envelopes) { @@ -576,6 +667,11 @@ async function main() { envelopes.push([chain, envelope]); await checkChainReadsAreServed(chain, envelope); } + if (chain !== 'bitcoin') { + // The dashboard families run whether or not the status answered: + // one red authority must not skip the rest of the matrix. + await checkChainDashboardFamilies(chain); + } } checkChainReleaseIdentity(envelopes); } diff --git a/scripts/universe/visual-qa/capture.mjs b/scripts/universe/visual-qa/capture.mjs index 1d6079f890..01f6da0f07 100644 --- a/scripts/universe/visual-qa/capture.mjs +++ b/scripts/universe/visual-qa/capture.mjs @@ -78,7 +78,11 @@ export const ROUTES = [ // The chain-domain routes. They shipped with no coverage here at all, which // is how eleven routes reached production without one screenshot, contrast // probe or unfinished-page check ever looking at them. - { id: 'dogecoin', path: '/dogecoin', name: 'Dogecoin overview' }, + { id: 'dogecoin', path: '/dogecoin', name: 'Dogecoin dashboard' }, + { id: 'dogecoin-mining', path: '/dogecoin/mining', name: 'Dogecoin mining' }, + { id: 'dogecoin-graphs', path: '/dogecoin/graphs/mempool', name: 'Dogecoin charts' }, + { id: 'dogecoin-graphs-pools', path: '/dogecoin/graphs/mining/pools', name: 'Dogecoin pool ranking' }, + { id: 'dogecoin-docs', path: '/dogecoin/docs', name: 'Dogecoin docs' }, { id: 'dogecoin-mempool', path: '/dogecoin/mempool', name: 'Dogecoin pending' }, { id: 'dogecoin-tx', path: `/dogecoin/tx/${chainSampleIds.DOGE_TXID}`, name: 'Dogecoin transaction' }, { id: 'dogecoin-block', path: `/dogecoin/block/${chainSampleIds.DOGE_BLOCK}`, name: 'Dogecoin block' }, @@ -88,7 +92,11 @@ export const ROUTES = [ // Dunes carry their own divisibility, so the pages exist to shift by it. { id: 'dogecoin-dunes', path: '/dogecoin/protocols/dunes', name: 'Dune catalog' }, { id: 'dogecoin-dune', path: `/dogecoin/protocols/dunes/${chainSampleIds.DOGE_DUNE_ID}`, name: 'Dune' }, - { id: 'zcash', path: '/zcash', name: 'Zcash overview' }, + { id: 'zcash', path: '/zcash', name: 'Zcash dashboard' }, + { id: 'zcash-mining', path: '/zcash/mining', name: 'Zcash mining' }, + { id: 'zcash-graphs', path: '/zcash/graphs/mempool', name: 'Zcash charts' }, + { id: 'zcash-graphs-hashrate', path: '/zcash/graphs/mining/hashrate-difficulty', name: 'Zcash hashrate chart' }, + { id: 'zcash-docs', path: '/zcash/docs', name: 'Zcash docs' }, { id: 'zcash-mempool', path: '/zcash/mempool', name: 'Zcash pending' }, { id: 'zcash-tx', path: `/zcash/tx/${chainSampleIds.ZEC_TXID}`, name: 'Zcash transaction' }, // Zcash's block response is not Dogecoin's, and nothing here had ever asked @@ -711,6 +719,10 @@ export const GATED_ROUTES = new Set([ 'dogecoin-dunes', 'dogecoin-dune', 'zcash', 'zcash-mempool', 'zcash-tx', 'zcash-protocols', 'zcash-zrc20', 'zcash-zrc20-token', + // The parity surfaces: dashboard timelines, mining, charts, and docs. The + // whole point of this release is that these exist and hold the same bar. + 'dogecoin-mining', 'dogecoin-graphs', 'dogecoin-graphs-pools', 'dogecoin-docs', + 'zcash-mining', 'zcash-graphs', 'zcash-graphs-hashrate', 'zcash-docs', // The single-asset pages and the local-state page, for the same reason. 'outpoint', 'inscription', 'rune', 'sat', 'saved', ]); diff --git a/scripts/universe/visual-qa/chain-fixtures.mjs b/scripts/universe/visual-qa/chain-fixtures.mjs index b1e72582e2..2a8e9ee769 100644 --- a/scripts/universe/visual-qa/chain-fixtures.mjs +++ b/scripts/universe/visual-qa/chain-fixtures.mjs @@ -663,7 +663,7 @@ export const chainFixtures = { snapshotId: 'dogecoin-mainnet-buckets', sequenceAtomic: '19', observedAt: new Date(Date.now() - 9_000).toISOString(), - tip: { heightAtomic: '5900001', blockHash: DOGE_BLOCK, observedAt: new Date(Date.now() - 9_000).toISOString() }, + tip: { heightAtomic: '5623041', blockHash: DOGE_BLOCK, observedAt: new Date(Date.now() - 9_000).toISOString() }, completeness: 'partial', semantics: 'ordered-by-fee-policy', feeModel: { kind: 'fee-per-kilobyte', unit: 'koinu/kB', minRelayFeeAtomicPerKb: '100000' }, @@ -697,7 +697,7 @@ export const chainFixtures = { snapshotId: 'zcash-buckets', sequenceAtomic: '22105', observedAt: new Date(Date.now() - 9_000).toISOString(), - tip: { heightAtomic: '3465589', blockHash: ZEC_BLOCK, observedAt: new Date(Date.now() - 9_000).toISOString() }, + tip: { heightAtomic: '3464729', blockHash: ZEC_BLOCK, observedAt: new Date(Date.now() - 9_000).toISOString() }, completeness: 'complete', semantics: 'zip317-eligibility-tiers', feeModel: { kind: 'zip-317', unit: 'zatoshi/logical-action', revisionAtomic: '1', marginalFeeAtomic: '5000', graceActionsAtomic: '2', unpaidActionLimitAtomic: null }, @@ -727,8 +727,306 @@ export const chainFixtures = { ], disclosures: ['zip317-random-selection', 'shielded-fee-unknown', 'unpaid-admission-not-modeled'], }, + + // The dashboard data families behind the chain dashboard, mining, and + // charts pages. Function declarations hoist, so the builders live at the + // bottom of this file with the other helpers. + '/api/v1/dogecoin/dashboard': dashboardFixture('dogecoin'), + '/api/v1/zcash/dashboard': dashboardFixture('zcash'), + '/api/v1/dogecoin/blocks/recent': recentBlocksFixture('dogecoin'), + '/api/v1/zcash/blocks/recent': recentBlocksFixture('zcash'), + '/api/v1/dogecoin/fees': feesFixture('dogecoin'), + '/api/v1/zcash/fees': feesFixture('zcash'), + '/api/v1/dogecoin/mining': miningSummaryFixture('dogecoin'), + '/api/v1/zcash/mining': miningSummaryFixture('zcash'), + '/api/v1/dogecoin/mining/pools': miningPoolsFixture('dogecoin'), + '/api/v1/zcash/mining/pools': miningPoolsFixture('zcash'), + ...chartSeriesFixtures('dogecoin'), + ...chartSeriesFixtures('zcash'), }; +// The dashboard aggregate carries the same buckets the bucket route serves, +// assigned after the table exists so there is one source for both spellings. +chainFixtures['/api/v1/dogecoin/dashboard'].buckets = + chainFixtures['/api/v1/dogecoin/candidate-buckets']; +chainFixtures['/api/v1/zcash/dashboard'].buckets = + chainFixtures['/api/v1/zcash/candidate-buckets']; + +/** The tip the dashboard fixtures agree on, matching the capability tip. */ +function fixtureTip(chain) { + const observedAt = new Date(Date.now() - 15_000).toISOString(); + return { + heightAtomic: chain === 'dogecoin' ? '5623041' : '3464729', + blockHash: chain === 'dogecoin' ? DOGE_BLOCK : ZEC_BLOCK, + observedAt, + }; +} + +/** Twelve stored blocks below the tip, with proven and unproven facts mixed. */ +function recentBlocksFixture(chain) { + const tip = fixtureTip(chain); + const tipHeight = Number(tip.heightAtomic); + const target = chain === 'dogecoin' ? 60 : 75; + const pools = chain === 'dogecoin' + ? [ + { poolId: 'viabtc', name: 'ViaBTC', evidence: 'auxpow-parent-tag' }, + { poolId: 'f2pool', name: 'F2Pool', evidence: 'payout-address' }, + null, + ] + : [ + { poolId: 'viabtc', name: 'ViaBTC', evidence: 'payout-address' }, + { poolId: '2miners', name: '2Miners', evidence: 'coinbase-tag' }, + null, + ]; + const blocks = []; + for (let index = 0; index < 12; index += 1) { + const height = tipHeight - index; + const pool = pools[index % pools.length]; + blocks.push({ + heightAtomic: String(height), + hash: `${height.toString(16)}`.padStart(64, chain === 'dogecoin' ? 'd' : '0'), + time: new Date(Date.now() - 15_000 - index * target * 1000 - (index % 3) * 11_000).toISOString(), + txCountAtomic: String(40 + ((index * 37) % 260)), + sizeBytesAtomic: chain === 'zcash' && index % 5 === 4 ? null : String(21_000 + ((index * 53_017) % 640_000)), + feesAtomic: index % 4 === 3 ? null : String(120_000_000 + ((index * 97_003_331) % 9_000_000_000)), + subsidyAtomic: chain === 'dogecoin' ? '1000000000000' : '156250000', + rewardAtomic: index % 4 === 3 ? null : String((chain === 'dogecoin' ? 1_000_000_000_000 : 156_250_000) + 120_000_000 + ((index * 97_003_331) % 9_000_000_000)), + medianFeeRateDecimal: chain === 'dogecoin' ? String(1_000_000 + ((index * 811_001) % 12_000_000)) : null, + difficultyDecimal: chain === 'dogecoin' ? '13648321.5' : '68227341.2', + intervalSecondsAtomic: index === 11 ? null : String(target + ((index * 29) % 44) - 20), + miner: pool + ? { poolId: pool.poolId, name: pool.name, evidence: pool.evidence } + : { poolId: null, name: null, evidence: null }, + }); + } + return { + schemaVersion: 'universe-recent-blocks-v1', + chain, + network: 'mainnet', + tip, + blocks, + coverage: { + fromHeightAtomic: String(tipHeight - 4_320), + toHeightAtomic: tip.heightAtomic, + complete: true, + }, + observedAt: new Date(Date.now() - 12_000).toISOString(), + }; +} + +function feesFixture(chain) { + const tip = fixtureTip(chain); + const observedAt = new Date(Date.now() - 12_000).toISOString(); + if (chain === 'dogecoin') { + return { + schemaVersion: 'universe-fee-recommendations-v1', + chain, + network: 'mainnet', + kind: 'fee-per-kilobyte', + unit: 'koinu/kB', + levels: [ + { id: 'none', amountDecimal: '100000', basis: 'relay-floor' }, + { id: 'low', amountDecimal: '1160000', basis: 'node-estimate' }, + { id: 'medium', amountDecimal: '5200000', basis: 'node-estimate' }, + { id: 'high', amountDecimal: '11190476', basis: 'node-estimate' }, + ], + minRelayFeeAtomicPerKb: '100000', + tip, + observedAt, + }; + } + return { + schemaVersion: 'universe-fee-recommendations-v1', + chain, + network: 'mainnet', + kind: 'zip-317', + unit: 'zatoshi', + marginalFeeAtomic: '5000', + graceActionsAtomic: '2', + typicalConventionalFeeAtomic: '10000', + paidShareDecimal: '0.75', + basis: 'zip317-conventional', + tip, + observedAt, + }; +} + +function miningSummaryFixture(chain) { + const tip = fixtureTip(chain); + const observedAt = new Date(Date.now() - 12_000).toISOString(); + return chain === 'dogecoin' + ? { + schemaVersion: 'universe-mining-summary-v1', + chain, + network: 'mainnet', + tip, + difficultyDecimal: '13648321.5', + networkRateDecimal: '976431000000000', + hashrateUnit: 'hashes-per-second', + algorithm: 'scrypt (AuxPoW merged mining)', + targetBlockSecondsAtomic: '60', + observedIntervalSecondsDecimal: '61.4', + windowBlocksAtomic: '1008', + subsidyAtomic: '1000000000000', + meanRewardAtomic: '1002481202210', + meanFeesAtomic: '2481202210', + mergedMining: { supported: true, noticeId: 'dogecoin-auxpow' }, + observedAt, + } + : { + schemaVersion: 'universe-mining-summary-v1', + chain, + network: 'mainnet', + tip, + difficultyDecimal: '68227341.2', + networkRateDecimal: '8123400000', + hashrateUnit: 'solutions-per-second', + algorithm: 'Equihash', + targetBlockSecondsAtomic: '75', + observedIntervalSecondsDecimal: '74.2', + windowBlocksAtomic: '960', + subsidyAtomic: '156250000', + meanRewardAtomic: '156329100', + meanFeesAtomic: '79100', + mergedMining: { supported: false, noticeId: null }, + observedAt, + }; +} + +function miningPoolsFixture(chain) { + const observedAt = new Date(Date.now() - 12_000).toISOString(); + const pools = chain === 'dogecoin' + ? [ + { poolId: 'viabtc', name: 'ViaBTC', blocksAtomic: '4183', shareDecimal: '0.415', evidence: ['auxpow-parent-tag'] }, + { poolId: 'f2pool', name: 'F2Pool', blocksAtomic: '2140', shareDecimal: '0.212', evidence: ['payout-address', 'auxpow-parent-tag'] }, + { poolId: 'antpool', name: 'AntPool', blocksAtomic: '1612', shareDecimal: '0.160', evidence: ['auxpow-parent-tag'] }, + { poolId: 'litecoinpool', name: 'Litecoinpool.org', blocksAtomic: '905', shareDecimal: '0.090', evidence: ['auxpow-parent-tag'] }, + { poolId: 'unknown', name: 'Unknown', blocksAtomic: '1240', shareDecimal: '0.123', evidence: [] }, + ] + : [ + { poolId: 'viabtc', name: 'ViaBTC', blocksAtomic: '3410', shareDecimal: '0.423', evidence: ['payout-address'] }, + { poolId: 'f2pool', name: 'F2Pool', blocksAtomic: '2120', shareDecimal: '0.263', evidence: ['payout-address'] }, + { poolId: '2miners', name: '2Miners', blocksAtomic: '1180', shareDecimal: '0.146', evidence: ['coinbase-tag'] }, + { poolId: 'unknown', name: 'Unknown', blocksAtomic: '1353', shareDecimal: '0.168', evidence: [] }, + ]; + return { + schemaVersion: 'universe-mining-pools-v1', + chain, + network: 'mainnet', + windowId: '1w', + windowBlocksAtomic: pools.reduce((sum, pool) => sum + Number(pool.blocksAtomic), 0).toString(), + pools, + attributionDatasetVersion: 'universe-pools-v1', + coverageComplete: true, + observedAt, + }; +} + +function dashboardFixture(chain) { + return { + schemaVersion: 'universe-chain-dashboard-v1', + chain, + network: 'mainnet', + tip: fixtureTip(chain), + recentBlocks: recentBlocksFixture(chain), + // Buckets are assigned after the fixture table exists; see the bottom + // of this file. + buckets: null, + fees: feesFixture(chain), + mempool: { + txCountAtomic: chain === 'dogecoin' ? '644' : '12', + totalSizeBytesAtomic: chain === 'dogecoin' ? '1489338' : '23620', + totalFeesAtomic: chain === 'dogecoin' ? null : '195500', + arrivalRatePerSecondDecimal: chain === 'dogecoin' ? '2.140' : '0.080', + observedAt: new Date(Date.now() - 12_000).toISOString(), + }, + mining: miningSummaryFixture(chain), + subsystems: [ + { id: 'core-node', state: 'ready', reasonIds: [] }, + { id: 'confirmed-history', state: 'ready', reasonIds: [] }, + { id: 'address-history', state: 'ready', reasonIds: [] }, + { id: 'mempool', state: 'ready', reasonIds: [] }, + { id: 'mining-analytics', state: 'ready', reasonIds: [] }, + { id: 'historical-statistics', state: 'ready', reasonIds: [] }, + { id: 'protocol-indexers', state: chain === 'dogecoin' ? 'degraded' : 'ready', reasonIds: chain === 'dogecoin' ? ['protocol-authority-stale'] : [] }, + ], + observedAt: new Date(Date.now() - 12_000).toISOString(), + }; +} + +/** One deterministic week of points per series, at one-hour resolution. */ +function chartSeriesFixtures(chain) { + // Listed here rather than at module scope: the fixture table calls this + // while the module is still initializing, before top-level consts exist. + const CHART_FIXTURE_SERIES = [ + 'mempool-count', 'mempool-size', 'mempool-fees', 'block-fees', + 'block-rewards', 'block-fees-subsidy', 'block-fee-rates', 'block-sizes', + 'block-count', 'block-interval', 'difficulty', 'hashrate', 'pools-dominance', + ]; + const entries = {}; + for (const seriesId of CHART_FIXTURE_SERIES) { + entries[`/api/v1/${chain}/charts/${seriesId}`] = chartSeriesFixture(chain, seriesId); + } + return entries; +} + +function chartSeriesFixture(chain, seriesId) { + const nowUnix = Math.floor(Date.now() / 1000); + const from = nowUnix - 7 * 86_400; + const points = (seed, scale, base) => { + const rows = []; + for (let hour = 0; hour < 168; hour += 1) { + const wobble = Math.abs(Math.sin(seed + hour / 9)) * scale; + rows.push([String(from + hour * 3_600), (base + wobble).toFixed(3)]); + } + return rows; + }; + const atomicUnit = chain === 'dogecoin' ? 'koinu' : 'zatoshi'; + const lines = { + 'mempool-count': [{ key: 'count', unit: 'transactions', points: points(1, 900, 120) }], + 'mempool-size': [{ key: 'size', unit: 'bytes', points: points(2, 2_400_000, 210_000) }], + 'mempool-fees': [{ key: 'fees', unit: atomicUnit, points: points(3, 9_000_000_000, 400_000_000) }], + 'block-fees': [{ key: 'fees', unit: atomicUnit, points: points(4, 8_000_000_000, 900_000_000) }], + 'block-rewards': [{ key: 'reward', unit: atomicUnit, points: points(5, 8_000_000_000, chain === 'dogecoin' ? 1_000_000_000_000 : 156_250_000) }], + 'block-fees-subsidy': [ + { key: 'fees', unit: atomicUnit, points: points(6, 8_000_000_000, 900_000_000) }, + { key: 'subsidy', unit: atomicUnit, points: points(0, 0, chain === 'dogecoin' ? 1_000_000_000_000 : 156_250_000) }, + ], + 'block-fee-rates': [{ key: 'median', unit: chain === 'dogecoin' ? 'koinu/kB' : 'zatoshi', points: points(7, 9_000_000, 1_000_000) }], + 'block-sizes': [{ key: 'size', unit: 'bytes', points: points(8, 500_000, 40_000) }], + 'block-count': [{ key: 'blocks', unit: 'blocks', points: points(9, 12, 54) }], + 'block-interval': [ + { key: 'observed', unit: 'seconds', points: points(10, 30, chain === 'dogecoin' ? 48 : 62) }, + { key: 'target', unit: 'seconds', points: points(0, 0, chain === 'dogecoin' ? 60 : 75) }, + ], + difficulty: [{ key: 'difficulty', unit: 'difficulty', points: points(11, 2_000_000, chain === 'dogecoin' ? 12_600_000 : 66_000_000) }], + hashrate: [{ key: 'rate', unit: chain === 'dogecoin' ? 'hashes-per-second' : 'solutions-per-second', points: points(12, 220_000_000_000_000, chain === 'dogecoin' ? 860_000_000_000_000 : 7_600_000_000) }], + 'pools-dominance': [ + { key: 'viabtc', unit: 'share', points: points(13, 0.08, 0.38) }, + { key: 'f2pool', unit: 'share', points: points(14, 0.06, 0.2) }, + { key: 'unknown', unit: 'share', points: points(15, 0.05, 0.12) }, + ], + }[seriesId]; + return { + schemaVersion: 'universe-chart-series-v1', + chain, + network: 'mainnet', + seriesId, + rangeId: '1w', + lines, + aggregation: seriesId.startsWith('mempool-') ? 'sample' : 'mean', + bucketSecondsAtomic: '3600', + coverage: { + fromAtomic: String(from), + toAtomic: String(nowUnix), + complete: true, + earliestAtomic: String(from - 21 * 86_400), + }, + sourceHeightAtomic: fixtureTip(chain).heightAtomic, + observedAt: new Date(Date.now() - 12_000).toISOString(), + }; +} + /** * Dogecoin as it stood on 2026-08-29: caught up on blocks, not ready overall, * because the protocol side is not answering and the DRC-20 index was created @@ -827,6 +1125,19 @@ export const chainStateOverrides = { '/api/v1/zcash/mempool': { status: 503 }, '/api/v1/dogecoin/candidate-buckets': { status: 503 }, '/api/v1/zcash/candidate-buckets': { status: 503 }, + // The dashboard families ride the same overlay, so the same outage + // reaches every one of them. The dashboard page has to degrade panel by + // panel under this, never blank the page. + '/api/v1/dogecoin/dashboard': { status: 503 }, + '/api/v1/zcash/dashboard': { status: 503 }, + '/api/v1/dogecoin/blocks/recent': { status: 503 }, + '/api/v1/zcash/blocks/recent': { status: 503 }, + '/api/v1/dogecoin/fees': { status: 503 }, + '/api/v1/zcash/fees': { status: 503 }, + '/api/v1/dogecoin/mining': { status: 503 }, + '/api/v1/zcash/mining': { status: 503 }, + '/api/v1/dogecoin/charts/': { status: 503 }, + '/api/v1/zcash/charts/': { status: 503 }, }, // The authority answers, and says it is behind. This is the state the whole diff --git a/scripts/universe/visual-qa/chain-page-audit.mjs b/scripts/universe/visual-qa/chain-page-audit.mjs index de5e75d544..db5650dd4e 100644 --- a/scripts/universe/visual-qa/chain-page-audit.mjs +++ b/scripts/universe/visual-qa/chain-page-audit.mjs @@ -311,3 +311,163 @@ export function auditChainPage({ chain, envelope, seen, observations = {} }) { return { failures, notes }; } + +/** + * The two sentences the timeline prints when a side is honestly empty. + * + * An empty side that says one of these is an early state, not a defect: the + * block collector has not caught up yet, or the pending set really is clear. + * An empty side that says nothing is absence dressed as data, and that is the + * defect these words tell apart. + */ +export const TIMELINE_EMPTY_COPY = { + confirmed: 'No recent blocks are stored yet', + future: 'Nothing is waiting right now', +}; + +/** + * What the rebuilt dashboard has to be true about, beyond what auditChainPage + * already holds it to. + * + * The state this exists to catch is a chain route that answers 200, renders a + * page, and the page is the capability report the dashboard replaced: no + * timeline, no panels, just headings about what the chain can answer. That + * page passes every network-level check, so the assertions here are + * structural: a timeline was drawn, its populated cubes name their heights, + * and an empty side explains itself in the timeline's own words rather than + * standing there bare. + */ +export function auditDashboardParity(chain, collected) { + const scope = `dashboard:${chain}`; + const failures = []; + const notes = []; + const fail = (message) => failures.push(`${scope}: ${message}`); + const pass = (message) => notes.push(`${scope}: ${message}`); + + const headings = collected.headings ?? []; + const timeline = collected.timeline ?? { present: false }; + + if (!timeline.present) { + // The two absences are told apart because they mean different things: the + // first is the old build still on the origin, the second is the new build + // losing its own timeline. + if (headings.some((heading) => heading.includes('Questions this chain can answer'))) { + fail( + 'the page leads with the capability report and draws no timeline, so the origin serves the page this dashboard replaced', + ); + } else { + fail('the block timeline region is absent, so this is not the rebuilt dashboard'); + } + } else { + const future = timeline.futureCubes ?? 0; + const confirmed = timeline.confirmedCubes ?? 0; + const emptySides = timeline.emptySides ?? []; + const explains = (copy) => emptySides.some((text) => text.includes(copy)); + if (future === 0 && confirmed === 0) { + fail('the timeline has zero cubes on both sides, which is a drawn strip with nothing behind it'); + } else { + pass(`timeline shows ${future} future slot(s) and ${confirmed} confirmed block(s)`); + for (const [side, count] of [['future', future], ['confirmed', confirmed]]) { + if (count > 0) { + continue; + } + if (explains(TIMELINE_EMPTY_COPY[side])) { + pass(`the ${side} side is empty and says so in its own words, an honest early state`); + } else { + fail(`the ${side} side is empty and gives no reason, so absence reads as data`); + } + } + if (timeline.heightsAboveCubes === false) { + fail('a populated cube carries no height label, so a reader cannot tell which block a cube stands for'); + } else { + pass('every populated cube carries its height label'); + } + } + if (timeline.hasDivider) { + pass('the timeline carries its divider'); + } + } + + const panels = collected.panels ?? []; + if (panels.length) { + pass(`panels read ${panels.join(', ')}`); + } + + const consoleErrors = collected.consoleErrors ?? []; + const failedRequests = collected.failedRequests ?? []; + if (consoleErrors.length) { + fail(`${consoleErrors.length} console error(s): ${consoleErrors.slice(0, 3).join(' | ')}`); + } + if (failedRequests.length) { + fail(`${failedRequests.length} failed request(s): ${failedRequests.slice(0, 3).join(' | ')}`); + } + + return { failures, notes }; +} + +/** The path segment each section lives under, so a redirect can be told from a render. */ +const SECTION_SEGMENTS = { mining: 'mining', graphs: 'graphs', docs: 'docs' }; + +/** + * What a section route has to be true about: it stayed where it was sent, it + * rendered its own content rather than a fallback, and it did so cleanly. + * + * The redirect check comes first because it explains the others. A mining + * route that bounces to the dashboard renders a perfectly healthy page with + * no mining heading, and without the final path the finding would blame the + * heading when the route is what moved. + */ +export function auditSectionPage(chain, section, collected) { + const scope = `${section}:${chain}`; + const failures = []; + const notes = []; + const fail = (message) => failures.push(`${scope}: ${message}`); + const pass = (message) => notes.push(`${scope}: ${message}`); + + const expected = `/${chain}/${SECTION_SEGMENTS[section] ?? section}`; + const finalPath = collected.finalPath ?? ''; + if (finalPath !== expected && !finalPath.startsWith(`${expected}/`)) { + fail( + `the browser ended at ${JSON.stringify(finalPath || null)} instead of under ${expected}, so the route redirected away`, + ); + } else { + pass(`stays at ${finalPath}`); + } + + const headings = collected.headings ?? []; + if (section === 'mining') { + if (headings.some((heading) => heading.toLowerCase().includes('mining'))) { + pass('renders its own mining heading'); + } else { + fail('no heading names mining, so the mining page did not render its own content'); + } + } else if (section === 'graphs') { + const links = collected.chartNavLinks ?? 0; + if (links > 0) { + pass(`the chart shell offers ${links} chart page link(s)`); + } else { + fail('the chart shell navigation is absent, so the graphs page did not render'); + } + } else if (section === 'docs') { + const sections = collected.docsSections ?? 0; + const links = collected.docsNavLinks ?? 0; + if (sections > 0 && links > 0) { + pass(`the docs render ${sections} section(s) behind a ${links}-entry section list`); + } else { + fail( + `the docs page renders ${sections} section(s) and ${links} section link(s), so the docs did not render`, + ); + } + } + + const consoleErrors = collected.consoleErrors ?? []; + const failedRequests = collected.failedRequests ?? []; + if (consoleErrors.length) { + fail(`${consoleErrors.length} console error(s): ${consoleErrors.slice(0, 3).join(' | ')}`); + } + if (failedRequests.length) { + fail(`${failedRequests.length} failed request(s): ${failedRequests.slice(0, 3).join(' | ')}`); + } + + return { failures, notes }; +} diff --git a/scripts/universe/visual-qa/chain-page-audit.test.mjs b/scripts/universe/visual-qa/chain-page-audit.test.mjs index 3997de4e65..4e87b693ba 100644 --- a/scripts/universe/visual-qa/chain-page-audit.test.mjs +++ b/scripts/universe/visual-qa/chain-page-audit.test.mjs @@ -13,7 +13,12 @@ import assert from 'node:assert/strict'; import test from 'node:test'; -import { auditChainPage, auditRelease } from './chain-page-audit.mjs'; +import { + auditChainPage, + auditDashboardParity, + auditRelease, + auditSectionPage, +} from './chain-page-audit.mjs'; const SNAPSHOT = 'dogecoin-mainnet-9304b4b3a53e788898da63a1b52cd509'; @@ -278,6 +283,176 @@ test('sideways scroll, console errors and third-party requests are detected', () assert.ok(saidSomethingAbout(result, 'third-party address(es)')); }); +/** The rebuilt dashboard as a healthy browser run collects it. */ +function dashboardCollected(overrides = {}) { + const { timeline, ...rest } = overrides; + return { + headings: ['Dogecoin dashboard', 'Block timeline', 'Transaction fees', 'Universe Lens', 'Mining'], + panels: [ + 'Block timeline', + 'Transaction fees', + 'Universe Lens', + 'Mining', + 'Waiting to confirm', + 'Recent blocks', + 'Arriving now', + 'Protocol indexers', + 'Subsystem health', + ], + consoleErrors: [], + failedRequests: [], + ...rest, + timeline: { + present: true, + futureCubes: 3, + confirmedCubes: 8, + heightsAboveCubes: true, + hasDivider: true, + emptySides: [], + ...(timeline ?? {}), + }, + }; +} + +/** A section page as a healthy browser run collects it. */ +function sectionCollected(section, overrides = {}) { + const base = { + mining: { headings: ['Dogecoin mining', 'Network', 'Mining pools'], finalPath: '/dogecoin/mining' }, + graphs: { headings: ['Dogecoin charts'], chartNavLinks: 10, finalPath: '/dogecoin/graphs/mempool' }, + docs: { + headings: ['Dogecoin explorer docs', 'Overview'], + docsNavLinks: 12, + docsSections: 12, + finalPath: '/dogecoin/docs', + }, + }[section]; + return { consoleErrors: [], failedRequests: [], ...base, ...overrides }; +} + +test('a healthy rebuilt dashboard passes', () => { + const result = auditDashboardParity('dogecoin', dashboardCollected()); + assert.deepEqual(result.failures, []); + assert.ok(result.notes.length >= 3); +}); + +test('a dashboard without a timeline is detected', () => { + const result = auditDashboardParity( + 'dogecoin', + dashboardCollected({ timeline: { present: false, futureCubes: 0, confirmedCubes: 0 } }), + ); + assert.ok(saidSomethingAbout(result, 'timeline region is absent')); +}); + +test('the capability report standing in for the dashboard is detected as the old build', () => { + const result = auditDashboardParity( + 'dogecoin', + dashboardCollected({ + headings: ['Questions this chain can answer', 'How much history is readable now'], + timeline: { present: false, futureCubes: 0, confirmedCubes: 0 }, + }), + ); + assert.ok(saidSomethingAbout(result, 'the page this dashboard replaced')); +}); + +test('a timeline with zero cubes on both sides is detected', () => { + const result = auditDashboardParity( + 'dogecoin', + dashboardCollected({ timeline: { futureCubes: 0, confirmedCubes: 0 } }), + ); + assert.ok(saidSomethingAbout(result, 'zero cubes on both sides')); +}); + +test('a populated cube missing its height label is detected', () => { + const result = auditDashboardParity( + 'dogecoin', + dashboardCollected({ timeline: { heightsAboveCubes: false } }), + ); + assert.ok(saidSomethingAbout(result, 'no height label')); +}); + +test('an empty confirmed side that explains itself passes with a note', () => { + const result = auditDashboardParity( + 'dogecoin', + dashboardCollected({ + timeline: { + confirmedCubes: 0, + emptySides: ['No recent blocks are stored yet. They appear as the collector catches up.'], + }, + }), + ); + assert.deepEqual(result.failures, []); + assert.ok(result.notes.some((note) => note.includes('confirmed side is empty and says so'))); +}); + +test('an empty side that gives no reason is detected', () => { + const result = auditDashboardParity( + 'dogecoin', + dashboardCollected({ timeline: { confirmedCubes: 0, emptySides: [] } }), + ); + assert.ok(saidSomethingAbout(result, 'gives no reason')); +}); + +test('console errors and failed answers on the dashboard are detected', () => { + const result = auditDashboardParity( + 'dogecoin', + dashboardCollected({ + consoleErrors: ['TypeError: undefined'], + failedRequests: ['/api/v1/dogecoin/dashboard HTTP 503'], + }), + ); + assert.ok(saidSomethingAbout(result, 'console error(s)')); + assert.ok(saidSomethingAbout(result, 'failed request(s)')); +}); + +test('each healthy section page passes', () => { + for (const section of ['mining', 'graphs', 'docs']) { + const result = auditSectionPage('dogecoin', section, sectionCollected(section)); + assert.deepEqual(result.failures, [], `${section} should pass clean`); + } +}); + +test('a mining page redirected to the dashboard is detected', () => { + // A route that bounces renders a perfectly healthy dashboard, so both the + // path and the missing heading have to be named. + const result = auditSectionPage( + 'dogecoin', + 'mining', + sectionCollected('mining', { + finalPath: '/dogecoin', + headings: ['Dogecoin dashboard', 'Transaction fees'], + }), + ); + assert.ok(saidSomethingAbout(result, 'redirected away')); + assert.ok(saidSomethingAbout(result, 'no heading names mining')); +}); + +test('a graphs page without its chart shell is detected', () => { + const result = auditSectionPage('dogecoin', 'graphs', sectionCollected('graphs', { chartNavLinks: 0 })); + assert.ok(saidSomethingAbout(result, 'chart shell navigation is absent')); +}); + +test('a docs page without its sections is detected', () => { + const result = auditSectionPage( + 'dogecoin', + 'docs', + sectionCollected('docs', { docsSections: 0, docsNavLinks: 0 }), + ); + assert.ok(saidSomethingAbout(result, 'docs did not render')); +}); + +test('console errors and failed answers on a section page are detected', () => { + const result = auditSectionPage( + 'dogecoin', + 'docs', + sectionCollected('docs', { + consoleErrors: ['ReferenceError: x is not defined'], + failedRequests: ['/api/v1/dogecoin/status HTTP 500'], + }), + ); + assert.ok(saidSomethingAbout(result, 'console error(s)')); + assert.ok(saidSomethingAbout(result, 'failed request(s)')); +}); + test('an origin serving a different build than the release names is detected', () => { assert.deepEqual(auditRelease('bea93c1ec', 'bea93c1ec7f608313').failures, []); assert.deepEqual(auditRelease('bea93c1ec', null).failures, []); diff --git a/scripts/universe/visual-qa/chain-page-smoke.mjs b/scripts/universe/visual-qa/chain-page-smoke.mjs index a55d5d1dd1..8ea39c4883 100644 --- a/scripts/universe/visual-qa/chain-page-smoke.mjs +++ b/scripts/universe/visual-qa/chain-page-smoke.mjs @@ -26,7 +26,13 @@ import { fileURLToPath } from 'node:url'; import playwright from 'playwright'; -import { CHAIN_NAMES, auditChainPage, auditRelease } from './chain-page-audit.mjs'; +import { + CHAIN_NAMES, + auditChainPage, + auditDashboardParity, + auditRelease, + auditSectionPage, +} from './chain-page-audit.mjs'; const HERE = dirname(fileURLToPath(import.meta.url)); const args = Object.fromEntries( @@ -122,6 +128,127 @@ function readPageText() { }; } +/** + * The facts the dashboard parity judgement needs: whether a timeline was + * drawn at all, how many cubes each side carries, whether every populated + * cube names its height, whether the divider is there, and what the empty + * sides say for themselves. + * + * Runs inside the page, so it may use no import from this file. + */ +function readDashboardFacts() { + const region = document.querySelector('.timeline-panel .timeline-row'); + const populated = [ + ...document.querySelectorAll('.timeline-side.future .candidate-cube'), + ...document.querySelectorAll('.timeline-side.confirmed .candidate-cube'), + ]; + return { + headings: [...document.querySelectorAll('h1, h2')].map((node) => node.textContent.trim()), + timeline: { + present: !!region, + futureCubes: document.querySelectorAll('.timeline-side.future .candidate-cube').length, + confirmedCubes: document.querySelectorAll('.timeline-side.confirmed .candidate-cube').length, + heightsAboveCubes: populated.every( + (cube) => (cube.querySelector('.cube-height')?.textContent ?? '').trim().length > 0, + ), + hasDivider: !!document.querySelector('.timeline-divider'), + emptySides: [...document.querySelectorAll('.timeline-side .timeline-empty')].map((node) => + node.textContent.trim(), + ), + }, + panels: [...document.querySelectorAll('.chain-page section.panel h2')].map((node) => + node.textContent.trim(), + ), + }; +} + +/** + * The facts a section page judgement needs. All three shapes are read on + * every section page and the judgement picks the one it is about, so this + * stays a single function the page can run. + * + * Runs inside the page, so it may use no import from this file. + */ +function readSectionFacts() { + return { + headings: [...document.querySelectorAll('h1, h2')].map((node) => node.textContent.trim()), + chartNavLinks: document.querySelectorAll('.graph-nav .graph-nav-link').length, + docsNavLinks: document.querySelectorAll('.docs-nav a').length, + docsSections: document.querySelectorAll('.docs-section').length, + }; +} + +/** + * One navigation, one screenshot, one set of facts. + * + * Each route gets its own page so console errors and failed answers are + * blamed on the page they happened on, not carried over from the previous + * route. A failed answer here means a same-origin server error or a missing + * API answer; a 404 on a document is the router's business and the app shell + * answers 200 for unknown routes anyway. + */ +async function visitAndCollect(context, path, screenshotName, reader) { + const page = await context.newPage(); + const consoleErrors = []; + const failedRequests = []; + page.on('console', (message) => { + if (message.type() === 'error') consoleErrors.push(message.text()); + }); + page.on('response', (response) => { + const url = new URL(response.url()); + if (url.origin !== new URL(ORIGIN).origin) { + return; + } + if (response.status() >= 500 || (response.status() === 404 && url.pathname.startsWith('/api/'))) { + failedRequests.push(`${response.url()} HTTP ${response.status()}`); + } + }); + try { + await page.goto(`${ORIGIN}${path}`, { waitUntil: 'networkidle', timeout: 45_000 }); + const facts = await page.evaluate(reader); + mkdirSync(OUT, { recursive: true }); + await page.screenshot({ path: join(OUT, screenshotName), fullPage: true }); + return { ...facts, consoleErrors, failedRequests, finalPath: new URL(page.url()).pathname }; + } finally { + await page.close(); + } +} + +/** + * The rebuilt dashboard and the section routes under it, one context per + * chain. This is additive to checkChain: that one holds the page to the + * capability document, this one holds the page to being the dashboard at + * all, and holds mining, graphs and docs to having rendered rather than + * having redirected somewhere that did. + */ +async function checkChainSections(browser, chain) { + const context = await browser.newContext({ viewport: { width: 1440, height: 900 } }); + try { + record( + auditDashboardParity( + chain, + await visitAndCollect(context, `/${chain}`, `${chain}-dashboard.png`, readDashboardFacts), + ), + ); + const sections = [ + ['mining', `/${chain}/mining`, `${chain}-mining.png`], + ['graphs', `/${chain}/graphs/mempool`, `${chain}-graphs-mempool.png`], + ['docs', `/${chain}/docs`, `${chain}-docs.png`], + ]; + for (const [section, path, screenshotName] of sections) { + record( + auditSectionPage( + chain, + section, + await visitAndCollect(context, path, screenshotName, readSectionFacts), + ), + ); + } + } finally { + await context.close(); + } +} + async function checkChain(browser, chain) { const { status, body: envelope } = await readJson(`/api/v1/${chain}/status`); if (status !== 200 || !envelope) { @@ -206,6 +333,7 @@ async function main() { try { for (const chain of Object.keys(CHAIN_NAMES)) { await checkChain(browser, chain); + await checkChainSections(browser, chain); } } finally { await browser.close(); diff --git a/scripts/universe/visual-qa/mobile-check.mjs b/scripts/universe/visual-qa/mobile-check.mjs index 68b5256b15..20200419f6 100644 --- a/scripts/universe/visual-qa/mobile-check.mjs +++ b/scripts/universe/visual-qa/mobile-check.mjs @@ -139,6 +139,9 @@ const FIELD_FLOOR = 16; const MOBILE_ROUTE_IDS = (args.routes ? String(args.routes).split(',') : [ 'home', 'tx', 'address', 'blocks', 'block', 'graphs', 'protocols', 'dogecoin-tx', 'zcash-block', 'docs', 'chain-menu', + // The parity surfaces, one thumb-width each: a chain dashboard with the + // block strip, a mining page, a chart page, and chain docs. + 'dogecoin', 'zcash-mining', 'dogecoin-graphs', 'zcash-docs', ]); const findings = []; From 6443ff2c38aecf2d13488153d094f85e330b4fbf Mon Sep 17 00:00:00 2001 From: Bitcoin Universe Date: Sun, 30 Aug 2026 14:10:02 +0000 Subject: [PATCH 03/15] Make the new chain controls thumb-sized on a phone The mobile gate measured the range, window, lens, and navigation pills at 31 pixels tall and as little as 39 wide, and the protocol and panel links at 24 pixels. All of them repeat in a row or a column, which is exactly the case the 44 pixel rule is about: a miss selects the neighbour. Each takes the house pattern already used by the protocol navigation: a coarse pointer or a narrow window gets a 44 pixel minimum in both directions. Desktop with a mouse is untouched. The two window rows that did not wrap now do, because a thumb-sized pill is wider than its label and the panel must not push sideways instead. Verified: mobile gate clean on all four new routes across seven window sizes; the visual matrix stays clean at 320, 375, and 1440 in both themes. Co-Authored-By: Claude Fable 5 --- .../chain-dashboard.component.scss | 17 +++++++++++ .../chain-dashboard/chain-lens.component.scss | 14 ++++++++++ .../chain-mining.component.scss | 27 ++++++++++++++++++ .../chain-docs/chain-docs.component.scss | 8 ++++++ .../chain-graphs/chain-chart.component.scss | 28 +++++++++++++++++++ .../chain-graphs/chain-graphs.component.scss | 14 ++++++++++ .../chain-pools-ranking.component.scss | 17 +++++++++++ 7 files changed, 125 insertions(+) diff --git a/frontend/src/app/universe/chain-dashboard/chain-dashboard.component.scss b/frontend/src/app/universe/chain-dashboard/chain-dashboard.component.scss index a7f7168350..18ceec1bb0 100644 --- a/frontend/src/app/universe/chain-dashboard/chain-dashboard.component.scss +++ b/frontend/src/app/universe/chain-dashboard/chain-dashboard.component.scss @@ -74,6 +74,15 @@ &:focus-visible { text-decoration: underline; } + + // One of these ends every panel, so on a phone they are a column of + // repeated targets. A line of text is about 24 pixels tall; 44 is the + // smallest a thumb can be expected to hit. + @media (pointer: coarse), (max-width: 991.98px) { + align-items: center; + display: inline-flex; + min-height: 44px; + } } } @@ -105,6 +114,14 @@ &:focus-visible { color: var(--u-brand); } + + // A column of one protocol per row, which on a phone is a stack of + // repeated targets a thumb has to pick between. + @media (pointer: coarse), (max-width: 991.98px) { + align-items: center; + display: inline-flex; + min-height: 44px; + } } } } diff --git a/frontend/src/app/universe/chain-dashboard/chain-lens.component.scss b/frontend/src/app/universe/chain-dashboard/chain-lens.component.scss index 011ddc6451..813240fe45 100644 --- a/frontend/src/app/universe/chain-dashboard/chain-lens.component.scss +++ b/frontend/src/app/universe/chain-dashboard/chain-lens.component.scss @@ -49,6 +49,20 @@ outline: 2px solid var(--u-focus-ring); outline-offset: 2px; } + + // A pill a finger has to hit. 44 is the smallest target a thumb can be + // expected to land on, and these repeat in a row where a miss selects the + // neighbour. + @media (pointer: coarse), (max-width: 991.98px) { + align-items: center; + display: inline-flex; + justify-content: center; + min-height: 44px; + // A label as short as "1w" is narrower than a thumb even when it is + // tall enough, so the target is square rather than only tall. + min-width: 44px; + padding-inline: var(--u-space-3); + } } .lens-table-toggle { diff --git a/frontend/src/app/universe/chain-dashboard/chain-mining.component.scss b/frontend/src/app/universe/chain-dashboard/chain-mining.component.scss index 90a7cba7c7..786d057ec3 100644 --- a/frontend/src/app/universe/chain-dashboard/chain-mining.component.scss +++ b/frontend/src/app/universe/chain-dashboard/chain-mining.component.scss @@ -59,6 +59,9 @@ .window-controls { display: flex; + // Thumb-sized pills are wider than the labels, so the row has to be + // allowed to wrap rather than push the panel sideways on a phone. + flex-wrap: wrap; gap: var(--u-space-2); margin-bottom: var(--u-space-3); } @@ -90,6 +93,20 @@ outline: 2px solid var(--u-focus-ring); outline-offset: 2px; } + + // A pill a finger has to hit. 44 is the smallest target a thumb can be + // expected to land on, and these repeat in a row where a miss selects the + // neighbour. + @media (pointer: coarse), (max-width: 991.98px) { + align-items: center; + display: inline-flex; + justify-content: center; + min-height: 44px; + // A label as short as "1w" is narrower than a thumb even when it is + // tall enough, so the target is square rather than only tall. + min-width: 44px; + padding-inline: var(--u-space-3); + } } .share-col { @@ -144,4 +161,14 @@ text-decoration: underline; } } + + a { + // A column of repeated targets on a phone. A line of text is about 24 + // pixels tall; 44 is the smallest a thumb can be expected to hit. + @media (pointer: coarse), (max-width: 991.98px) { + align-items: center; + display: inline-flex; + min-height: 44px; + } + } } diff --git a/frontend/src/app/universe/chain-docs/chain-docs.component.scss b/frontend/src/app/universe/chain-docs/chain-docs.component.scss index 2b9fa08124..309b1d74d6 100644 --- a/frontend/src/app/universe/chain-docs/chain-docs.component.scss +++ b/frontend/src/app/universe/chain-docs/chain-docs.component.scss @@ -77,6 +77,14 @@ font-size: var(--u-text-sm); text-decoration: none; + // A list of repeated targets, which on a phone is exactly the case the + // 44 pixel rule is about. + @media (pointer: coarse), (max-width: 991.98px) { + align-items: center; + display: flex; + min-height: 44px; + } + &:hover { color: var(--u-brand); } diff --git a/frontend/src/app/universe/chain-graphs/chain-chart.component.scss b/frontend/src/app/universe/chain-graphs/chain-chart.component.scss index bb25e87491..5e21023435 100644 --- a/frontend/src/app/universe/chain-graphs/chain-chart.component.scss +++ b/frontend/src/app/universe/chain-graphs/chain-chart.component.scss @@ -49,6 +49,20 @@ outline: 2px solid var(--u-focus-ring); outline-offset: 2px; } + + // A pill a finger has to hit. 44 is the smallest target a thumb can be + // expected to land on, and these repeat in a row where a miss selects the + // neighbour. + @media (pointer: coarse), (max-width: 991.98px) { + align-items: center; + display: inline-flex; + justify-content: center; + min-height: 44px; + // A label as short as "1w" is narrower than a thumb even when it is + // tall enough, so the target is square rather than only tall. + min-width: 44px; + padding-inline: var(--u-space-3); + } } .chart-card { @@ -88,6 +102,20 @@ outline: 2px solid var(--u-focus-ring); outline-offset: 2px; } + + // A pill a finger has to hit. 44 is the smallest target a thumb can be + // expected to land on, and these repeat in a row where a miss selects the + // neighbour. + @media (pointer: coarse), (max-width: 991.98px) { + align-items: center; + display: inline-flex; + justify-content: center; + min-height: 44px; + // A label as short as "1w" is narrower than a thumb even when it is + // tall enough, so the target is square rather than only tall. + min-width: 44px; + padding-inline: var(--u-space-3); + } } .chart { diff --git a/frontend/src/app/universe/chain-graphs/chain-graphs.component.scss b/frontend/src/app/universe/chain-graphs/chain-graphs.component.scss index 67b7cc8445..1a20dd4147 100644 --- a/frontend/src/app/universe/chain-graphs/chain-graphs.component.scss +++ b/frontend/src/app/universe/chain-graphs/chain-graphs.component.scss @@ -81,4 +81,18 @@ outline: 2px solid var(--u-focus-ring); outline-offset: 2px; } + + // A pill a finger has to hit. 44 is the smallest target a thumb can be + // expected to land on, and these repeat in a row where a miss selects the + // neighbour. + @media (pointer: coarse), (max-width: 991.98px) { + align-items: center; + display: inline-flex; + justify-content: center; + min-height: 44px; + // A label as short as "1w" is narrower than a thumb even when it is + // tall enough, so the target is square rather than only tall. + min-width: 44px; + padding-inline: var(--u-space-3); + } } diff --git a/frontend/src/app/universe/chain-graphs/chain-pools-ranking.component.scss b/frontend/src/app/universe/chain-graphs/chain-pools-ranking.component.scss index 410233c8d6..5817cddacb 100644 --- a/frontend/src/app/universe/chain-graphs/chain-pools-ranking.component.scss +++ b/frontend/src/app/universe/chain-graphs/chain-pools-ranking.component.scss @@ -24,6 +24,9 @@ .window-controls { display: flex; + // Thumb-sized pills are wider than the labels, so the row wraps rather + // than pushing the panel sideways on a phone. + flex-wrap: wrap; gap: var(--u-space-2); margin-bottom: var(--u-space-3); } @@ -55,6 +58,20 @@ outline: 2px solid var(--u-focus-ring); outline-offset: 2px; } + + // A pill a finger has to hit. 44 is the smallest target a thumb can be + // expected to land on, and these repeat in a row where a miss selects the + // neighbour. + @media (pointer: coarse), (max-width: 991.98px) { + align-items: center; + display: inline-flex; + justify-content: center; + min-height: 44px; + // A label as short as "1w" is narrower than a thumb even when it is + // tall enough, so the target is square rather than only tall. + min-width: 44px; + padding-inline: var(--u-space-3); + } } .donut-figure { From 9ff8a50fbdf0196e7324a68da7551bfc687d54d7 Mon Sep 17 00:00:00 2001 From: Bitcoin Universe Date: Sun, 30 Aug 2026 16:43:19 +0000 Subject: [PATCH 04/15] Ask each chain for a pending set it will actually return Dogecoin returns up to a thousand pending transactions in one request. Zcash refuses anything above two hundred as a bad request rather than trimming it. The dashboard asked both for four hundred, so the Zcash lens and its arrivals list were empty on every real load while every fixture and every unit test passed: a fixture answers whatever it is asked. The ceilings now live beside the request that has to respect them and are clamped where the URL is built, so no caller can exceed one. Two things follow from finding it this way: - live-e2e.mjs is added. It loads the local build and forwards only its API calls to a deployed origin, which is the cheapest check that can see a contract the two sides disagree about. It reports rather than asserts, because the deployment it reads is live and a chain in a genuine outage would fail assertions for telling the truth. - The fault panel's own styling moved into the shared chain-page partial. It had stayed behind in the old component, so the dashboard rendered "Why Dogecoin is not ready" at browser-default heading size, outranking the page title. Verified against the live origin: both dashboards render the lens, the timeline and every panel with no failure notice, over real blocks and real pool attribution. Co-Authored-By: Claude Fable 5 --- frontend/src/app/universe/_chain-page.scss | 34 +++++ .../multichain-explorer.component.scss | 32 +---- .../app/universe/universe-api.service.spec.ts | 25 ++++ .../src/app/universe/universe-api.service.ts | 17 ++- scripts/universe/visual-qa/live-e2e.mjs | 132 ++++++++++++++++++ 5 files changed, 209 insertions(+), 31 deletions(-) create mode 100644 scripts/universe/visual-qa/live-e2e.mjs diff --git a/frontend/src/app/universe/_chain-page.scss b/frontend/src/app/universe/_chain-page.scss index 6ae9b1e9da..db4be77251 100644 --- a/frontend/src/app/universe/_chain-page.scss +++ b/frontend/src/app/universe/_chain-page.scss @@ -127,6 +127,40 @@ color: var(--u-text-muted); } + // Why a chain is not ready. Bordered like a notice rather than raised like + // a panel: it qualifies the rail above it rather than opening a section, + // and its heading must not outrank the page's own. + .not-ready { + @include u.universe-surface; + + border-left: 3px solid var(--universe-state-partial); + padding: var(--u-space-3) var(--u-space-4); + margin-bottom: var(--u-space-3); + max-width: var(--u-width-prose); + + h2 { + font-size: var(--u-text-base); + font-weight: var(--u-weight-semibold); + margin: 0 0 var(--u-space-2); + } + + ul { + display: grid; + gap: var(--u-space-2); + font-size: var(--u-text-sm); + line-height: var(--u-leading-normal); + list-style: none; + margin: 0; + padding: 0; + } + } + + .reason-raw { + color: var(--u-text-muted); + display: block; + font-size: var(--u-text-xs); + } + .identifier { @include u.universe-identifier; } diff --git a/frontend/src/app/universe/multichain-explorer/multichain-explorer.component.scss b/frontend/src/app/universe/multichain-explorer/multichain-explorer.component.scss index 045c22d131..c46cc5e524 100644 --- a/frontend/src/app/universe/multichain-explorer/multichain-explorer.component.scss +++ b/frontend/src/app/universe/multichain-explorer/multichain-explorer.component.scss @@ -72,36 +72,8 @@ // Sits directly beneath the rail, because it is the evidence for the verdict // the rail just gave. Bordered like a notice rather than raised like a panel: // it is a qualification of what is above it, not a section of its own. -.not-ready { - @include u.universe-surface; - - border-left: 3px solid var(--universe-state-partial); - padding: var(--u-space-3) var(--u-space-4); - margin-bottom: var(--u-space-3); - max-width: var(--u-width-prose); - - h2 { - font-size: var(--u-text-base); - font-weight: var(--u-weight-semibold); - margin: 0 0 var(--u-space-2); - } - - ul { - display: grid; - gap: var(--u-space-2); - font-size: var(--u-text-sm); - line-height: var(--u-leading-normal); - list-style: none; - margin: 0; - padding: 0; - } -} - -.reason-raw { - color: var(--u-text-muted); - display: block; - font-size: var(--u-text-xs); -} +// The fault panel and its raw-code line are in the shared chain-page +// partial, so every chain page states a fault the same way. // --------------------------------------------------------------------------- // Readiness band diff --git a/frontend/src/app/universe/universe-api.service.spec.ts b/frontend/src/app/universe/universe-api.service.spec.ts index f76cd6c7e6..8e3925f72d 100644 --- a/frontend/src/app/universe/universe-api.service.spec.ts +++ b/frontend/src/app/universe/universe-api.service.spec.ts @@ -90,6 +90,31 @@ describe('UniverseApiService addressing', () => { }); }); +describe('UniverseApiService pending-set bounds', () => { + // Zcash refuses a limit above 200 as a bad request rather than trimming it, + // so a shared default emptied its lens and its arrivals list in production + // while every fixture answered whatever it was asked. + it('never asks a chain for more pending transactions than it allows', () => { + const { service, urls } = build(true); + service.getChainMempool$('zcash', 400).subscribe(); + service.getChainMempool$('dogecoin', 400).subscribe(); + expect(urls[0]).toContain('limit=200'); + expect(urls[1]).toContain('limit=400'); + }); + + it('keeps a request for fewer than the ceiling exactly as asked', () => { + const { service, urls } = build(true); + service.getChainMempool$('zcash', 50).subscribe(); + expect(urls[0]).toContain('limit=50'); + }); + + it('refuses a limit below one', () => { + const { service, urls } = build(true); + service.getChainMempool$('dogecoin', 0).subscribe(); + expect(urls[0]).toContain('limit=1'); + }); +}); + describe('UniverseApiService protocol registry cache', () => { it('fetches the registry once and replays it', () => { const get = vi.fn(() => of({ registryVersion: '1.0.0' })); diff --git a/frontend/src/app/universe/universe-api.service.ts b/frontend/src/app/universe/universe-api.service.ts index 40282a55b7..d159086a83 100644 --- a/frontend/src/app/universe/universe-api.service.ts +++ b/frontend/src/app/universe/universe-api.service.ts @@ -32,6 +32,20 @@ import { export const UNIVERSE_OUTPOINT_BATCH_LIMIT = 50; export const UNIVERSE_TRANSACTION_BATCH_LIMIT = 25; +/** + * How many pending transactions each chain will return in one request. + * + * The two chains do not share a ceiling, and asking for more than a chain + * allows is refused as a bad request rather than trimmed. Asking Zcash for + * four hundred emptied its lens and its arrivals list in production while + * every fixture answered whatever it was asked, so the bound is enforced + * here, once, where the request is built. + */ +export const CHAIN_MEMPOOL_LIMIT: Record, number> = { + dogecoin: 1000, + zcash: 200, +}; + @Injectable({ providedIn: 'root' }) @@ -166,7 +180,8 @@ export class UniverseApiService { getChainMempool$(chain: Exclude, limit = 100): Observable { return this.httpClient.get( - this.apiBaseUrl + '/api/v1/' + chain + '/mempool?network=mainnet&limit=' + limit + this.apiBaseUrl + '/api/v1/' + chain + '/mempool?network=mainnet&limit=' + + Math.min(Math.max(1, Math.floor(limit)), CHAIN_MEMPOOL_LIMIT[chain]) ); } diff --git a/scripts/universe/visual-qa/live-e2e.mjs b/scripts/universe/visual-qa/live-e2e.mjs new file mode 100644 index 0000000000..c4f77b33d2 --- /dev/null +++ b/scripts/universe/visual-qa/live-e2e.mjs @@ -0,0 +1,132 @@ +#!/usr/bin/env node +/** + * The locally built frontend, loaded against the live API. + * + * The matrix beside this one answers every request from fixtures, which is + * what makes it fast and deterministic and also what makes it blind to a + * bound the real service enforces. A fixture answers whatever it is asked. + * A chain does not: Dogecoin returns up to a thousand pending transactions + * and Zcash refuses anything above two hundred as a bad request rather than + * trimming it. The dashboard asked both for four hundred, so the Zcash lens + * and arrivals list were empty on every real load while every fixture and + * every unit test passed. + * + * So this loads the build that is about to ship and sends its API calls to + * a deployed origin. It is the cheapest check that can see a contract the + * two sides disagree about, and it belongs between a green matrix and a + * cutover rather than after one. + * + * It reports what each page rendered rather than asserting: the deployment + * it reads is live, so a chain in a genuine outage would fail assertions + * for telling the truth. Read the output, and the screenshots beside it. + * + * Usage: + * node live-e2e.mjs [--local=URL] [--live=URL] [--out=DIR] ... + * node live-e2e.mjs dogecoin zcash dogecoin/mining zcash/graphs/mempool + */ +import { mkdirSync } from 'node:fs'; +import { dirname, join, resolve } from 'node:path'; +import { fileURLToPath } from 'node:url'; + +import playwright from 'playwright'; + +const HERE = dirname(fileURLToPath(import.meta.url)); +const args = []; +const options = {}; +for (const argument of process.argv.slice(2)) { + const match = /^--([a-z-]+)=(.*)$/.exec(argument); + if (match) { + options[match[1]] = match[2]; + } else { + args.push(argument.replace(/^\/+/, '')); + } +} + +const LOCAL = (options.local || 'http://127.0.0.1:8123').replace(/\/+$/, ''); +const LIVE = (options.live || 'https://explorer.bitcoinuniverse.io').replace(/\/+$/, ''); +const OUT = resolve(options.out || join(HERE, 'artifacts-live')); +const PATHS = args.length ? args : ['dogecoin', 'zcash']; + +mkdirSync(OUT, { recursive: true }); + +const browser = await playwright.chromium.launch(); +const context = await browser.newContext({ viewport: { width: 1440, height: 900 } }); + +// Only the API is forwarded. The page, its chunks and its assets come from +// the local build, which is the half being tested. +await context.route('**/api/**', async (route) => { + const url = new URL(route.request().url()); + try { + const response = await fetch(LIVE + url.pathname + url.search, { + headers: { accept: 'application/json' }, + signal: AbortSignal.timeout(20_000), + }); + await route.fulfill({ + status: response.status, + contentType: response.headers.get('content-type') || 'application/json', + body: await response.text(), + }); + } catch { + await route.fulfill({ + status: 503, + contentType: 'application/json', + body: JSON.stringify({ error: 'live-origin-unreachable' }), + }); + } +}); + +/** + * What a chain page rendered. Counted rather than judged, for the reason in + * the header: this reads a live deployment. + */ +function readPage() { + const text = (node) => node.textContent.trim(); + return { + h1: document.querySelector('h1')?.textContent?.trim() ?? null, + panels: [...document.querySelectorAll('section.panel h2')].map(text).slice(0, 12), + futureCubes: document.querySelectorAll('.timeline-side.future .candidate-cube').length, + confirmedCubes: document.querySelectorAll('.timeline-side.confirmed .candidate-cube').length, + heights: [...document.querySelectorAll('.cube-height')].slice(0, 6).map(text), + miners: [...document.querySelectorAll('.miner-name')].slice(0, 4).map(text), + charts: document.querySelectorAll('div[_echarts_instance_]').length, + lensRects: document.querySelectorAll('canvas.block-overview-canvas').length, + notices: [...document.querySelectorAll('.notice.empty, .notice.failure')] + .map((node) => text(node).slice(0, 80)) + .slice(0, 5), + }; +} + +console.log(`local ${LOCAL}, API from ${LIVE}`); +for (const path of PATHS) { + const page = await context.newPage(); + const errors = []; + page.on('console', (message) => { + // A websocket to the local gateway has no upstream here and its refusal + // is a fact about this harness, not about the build. + const text = message.text(); + if (message.type() === 'error' && !/WebSocket connection to/.test(text)) { + errors.push(text.slice(0, 140)); + } + }); + page.on('pageerror', (error) => errors.push(`pageerror: ${String(error).slice(0, 140)}`)); + try { + await page.goto(`${LOCAL}/${path}`, { waitUntil: 'networkidle', timeout: 60_000 }); + await page.waitForTimeout(3_500); + const facts = await page.evaluate(readPage); + await page.screenshot({ + path: join(OUT, `${path.replace(/[^a-z0-9]+/gi, '_') || 'root'}.png`), + }); + console.log(`\n--- /${path}`); + console.log(JSON.stringify(facts, null, 1)); + } catch (error) { + console.log(`\n--- /${path}`); + console.log(` did not load: ${String(error).slice(0, 140)}`); + } + if (errors.length) { + console.log(' console errors:', errors.slice(0, 4)); + } + await page.close(); +} + +await browser.close(); +console.log(`\nscreenshots in ${OUT}`); From 2b0e4989ffed92fb89dc8a12bc3f93cb123c6372 Mon Sep 17 00:00:00 2001 From: Bitcoin Universe Date: Sun, 30 Aug 2026 18:29:02 +0000 Subject: [PATCH 05/15] Hold the protocol roster to the registry that owns it The gate this replaces rendered a Markdown table from the recorded JSON and compared it against the committed Markdown, so the only thing it could ever prove was that a file agreed with itself. While it was green, production served thirty-eight protocols and the recorded roster here said thirty-six: dunes and zrc20 were added to the registry in backend-apis, served publicly, and nothing in this repository noticed. The README was wrong in the same way, naming three readable protocols for as long as six had been readable. The roster is owned by backend-apis, which now publishes it as a named document: manifest schema, registry version, the repository that owns it and the commit it was built from. This pins that document and holds four things to it. --check, offline, on every commit: the pin carries its provenance, or it is not a pin; the readable list in the README and the coverage table are generated from it; PROTOCOL-ROSTER.lock records every published id, so a protocol leaving the roster has to be an edit rather than an absence; every protocol this repository writes prose for or calls an API path for exists in the registry, on the chain the registry places it on, and every readable protocol has a path. --against , in the production smoke run: the served roster is compared field by field against the pin, and a protocol added, dropped, renamed, re-authorised or moved between chains is named rather than discovered later by a reader. Twenty-three cases in protocol-contract.test.mjs, each one a state the old gate was green for. Co-Authored-By: Claude Opus 5 --- .github/workflows/universe-ci.yml | 14 +- .../workflows/universe-production-smoke.yml | 8 + CONTRIBUTING.md | 2 +- README.md | 25 +- docs/protocols/PROTOCOL-COVERAGE.json | 59 +- docs/protocols/PROTOCOL-COVERAGE.md | 30 +- docs/protocols/PROTOCOL-ROSTER.lock | 41 + .../universe/generate-protocol-coverage.mjs | 193 ----- scripts/universe/protocol-contract.mjs | 761 ++++++++++++++++++ scripts/universe/protocol-contract.test.mjs | 267 ++++++ 10 files changed, 1183 insertions(+), 217 deletions(-) create mode 100644 docs/protocols/PROTOCOL-ROSTER.lock delete mode 100644 scripts/universe/generate-protocol-coverage.mjs create mode 100644 scripts/universe/protocol-contract.mjs create mode 100644 scripts/universe/protocol-contract.test.mjs diff --git a/.github/workflows/universe-ci.yml b/.github/workflows/universe-ci.yml index a71293aa03..cc2f76318c 100644 --- a/.github/workflows/universe-ci.yml +++ b/.github/workflows/universe-ci.yml @@ -44,8 +44,18 @@ jobs: with: node-version: '24.19.0' - - name: Protocol coverage table matches the recorded manifest - run: node scripts/universe/generate-protocol-coverage.mjs --check + # The roster is owned by backend-apis. This holds the pinned copy to its + # schema and its provenance, holds this repository's own surfaces to the + # pinned roster, and refuses a roster that lost an entry. The gate it + # replaced rendered a table from the recorded JSON and compared it with + # the committed table, so it could only prove a file agreed with itself: + # dunes and zrc20 were added to the registry and served in production + # while the recorded roster here still said thirty-six, green throughout. + - name: The pinned protocol roster holds + run: node scripts/universe/protocol-contract.mjs --check + + - name: The protocol roster gate fires on the states it exists for + run: node --test scripts/universe/protocol-contract.test.mjs - name: No em dash anywhere run: node scripts/universe/check-text.mjs diff --git a/.github/workflows/universe-production-smoke.yml b/.github/workflows/universe-production-smoke.yml index 08d43340d1..9669dccd1a 100644 --- a/.github/workflows/universe-production-smoke.yml +++ b/.github/workflows/universe-production-smoke.yml @@ -43,6 +43,14 @@ jobs: with: node-version: '24.19.0' + # The pinned roster is a copy of another repository's registry, and a copy + # is only worth anything while something compares it. This is that + # comparison, against what the deployment actually serves: a protocol + # added, dropped, renamed, re-authorised or moved between chains fails + # here rather than being discovered by a reader of a stale table. + - name: The served protocol roster matches the pinned one + run: node scripts/universe/protocol-contract.mjs --against "${{ github.event.inputs.origin || 'https://explorer.bitcoinuniverse.io' }}" + - name: Every advertised feature has routes, data, and a matching release run: node scripts/universe/synthetic-check.mjs "${{ github.event.inputs.origin || 'https://explorer.bitcoinuniverse.io' }}" diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 27c99b202a..9888a496ab 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -33,7 +33,7 @@ Run these before opening a pull request. They are the same ones ```bash cd frontend && npm ci && npm run lint && npm run test && npm run build:universe cd backend && npm ci && npm run lint && npm run test:ci -node scripts/universe/generate-protocol-coverage.mjs --check +node scripts/universe/protocol-contract.mjs --check node scripts/universe/check-text.mjs node scripts/universe/check-colors.mjs node scripts/universe/check-palettes.mjs diff --git a/README.md b/README.md index 2cee2d30e2..13808f48e8 100644 --- a/README.md +++ b/README.md @@ -111,8 +111,14 @@ explorer states plainly which ones it can actually read. | Live, read only | A first-party authority is running and its evidence is shown. | | Not yet available | No first-party authority for it is configured or answering here. The explorer makes no claim about it. | -Readable today, backed by the first-party Ord 0.29 authority: **Ordinals**, -**Rare Sats**, **Runes**. + + +6 of the 38 protocols in the registry are readable today: + +- On bitcoin: **Ordinals**, **Rare Sats**, **Runes**, from ord. +- On zcash: **Zerdinals**, **ZRunes**, **ZRC-20**, from index-zcash-metaprotocols. + + Readable is not the same as current, and the product never conflates them. An authority that is rebuilding its index is still answering, and every page that @@ -121,10 +127,15 @@ answers as the present. `/api/v1/universe/sources` publishes the same figure, and the production smoke check in `.github/workflows/universe-production-smoke.yml` reads it on a schedule. -`docs/protocols/PROTOCOL-COVERAGE.md` is generated from the registry and lists -every entry with its authority and its release status. Run -`node scripts/universe/generate-protocol-coverage.mjs --check` to verify the -table still matches the recorded manifest. +The roster itself is owned by `bitcoinuniverseio/backend-apis` and served by +`/api/v1/universe/protocols`. This repository pins a copy of it in +`docs/protocols/PROTOCOL-COVERAGE.json`, carrying the schema, the registry +version, the repository that produced it and the commit it was produced from, +and `docs/protocols/PROTOCOL-COVERAGE.md` is the readable table. +`node scripts/universe/protocol-contract.mjs --check` holds the pinned roster +and this repository's own surfaces together, and +`node scripts/universe/protocol-contract.mjs --against https://explorer.bitcoinuniverse.io` +fails when a deployment serves a roster that differs from the pin. ## Architecture @@ -222,7 +233,7 @@ cd frontend && npm run build:universe # production build, no third-party fetch cd frontend && npm run test # Universe unit suite cd frontend && npm run lint cd backend && npm run test:ci && npm run lint -node scripts/universe/generate-protocol-coverage.mjs --check +node scripts/universe/protocol-contract.mjs --check node scripts/universe/check-text.mjs # no em dash anywhere node scripts/universe/check-colors.mjs # no raw interface colour node scripts/universe/check-palettes.mjs # measured contrast, both themes diff --git a/docs/protocols/PROTOCOL-COVERAGE.json b/docs/protocols/PROTOCOL-COVERAGE.json index 52c0383c6d..e155acb089 100644 --- a/docs/protocols/PROTOCOL-COVERAGE.json +++ b/docs/protocols/PROTOCOL-COVERAGE.json @@ -1,6 +1,17 @@ { + "schemaVersion": "universe-explorer-protocol-manifest-v1", "registryVersion": "1.0.0", - "generatedAt": "2026-08-26T06:40:08.985Z", + "sourceRepository": "bitcoinuniverseio/backend-apis", + "sourceSha": "88a616cfdabc93facd3dcc3296f88dc214c24a40", + "generatedAt": "2026-08-30T18:18:38.168Z", + "primaryStrip": [ + "ORDINALS", + "RUNES", + "ALKANES", + "STAMPS", + "ATOMICALS", + "MORE" + ], "protocols": [ { "schemaVersion": "universe-explorer-protocol-v1", @@ -730,6 +741,25 @@ "indexerAuthority": "index-doge-tap", "coverage": "unknown" }, + { + "schemaVersion": "universe-explorer-protocol-v1", + "id": "dunes", + "aliases": [], + "displayName": "Dunes", + "shortName": "Dunes", + "family": "OTHER", + "chain": "dogecoin", + "networks": [ + "mainnet" + ], + "icon": "protocol-dunes", + "visualToken": "protocol-dunes", + "implementedReadOperations": [], + "authorizedReadOperations": [], + "releaseStatus": "BLOCKED", + "indexerAuthority": "ord-dogecoin", + "coverage": "unknown" + }, { "schemaVersion": "universe-explorer-protocol-v1", "id": "zerdinals", @@ -747,9 +777,9 @@ "visualToken": "protocol-zerdinals", "implementedReadOperations": [], "authorizedReadOperations": [], - "releaseStatus": "BLOCKED", + "releaseStatus": "VERIFIED READ ONLY", "indexerAuthority": "index-zcash-metaprotocols", - "coverage": "unknown" + "coverage": "complete" }, { "schemaVersion": "universe-explorer-protocol-v1", @@ -766,9 +796,28 @@ "visualToken": "protocol-zrunes", "implementedReadOperations": [], "authorizedReadOperations": [], - "releaseStatus": "BLOCKED", + "releaseStatus": "VERIFIED READ ONLY", "indexerAuthority": "index-zcash-metaprotocols", - "coverage": "unknown" + "coverage": "complete" + }, + { + "schemaVersion": "universe-explorer-protocol-v1", + "id": "zrc20", + "aliases": [], + "displayName": "ZRC-20", + "shortName": "ZRC-20", + "family": "OTHER", + "chain": "zcash", + "networks": [ + "mainnet" + ], + "icon": "protocol-zrc20", + "visualToken": "protocol-zrc20", + "implementedReadOperations": [], + "authorizedReadOperations": [], + "releaseStatus": "VERIFIED READ ONLY", + "indexerAuthority": "index-zcash-metaprotocols", + "coverage": "complete" } ] } diff --git a/docs/protocols/PROTOCOL-COVERAGE.md b/docs/protocols/PROTOCOL-COVERAGE.md index 8b832095a5..6281aa4bd5 100644 --- a/docs/protocols/PROTOCOL-COVERAGE.md +++ b/docs/protocols/PROTOCOL-COVERAGE.md @@ -1,20 +1,30 @@ # Protocol coverage -Generated from the explorer protocol registry in backend-apis -(src/universe-explorer/registry/explorer-protocol-registry.ts), as served by -`/api/v1/universe/protocols`. Do not edit rows by hand: regenerate with +The roster is owned by `bitcoinuniverseio/backend-apis`, in +`src/universe-explorer/registry/explorer-protocol-registry.ts`, and served by +`/api/v1/universe/protocols`. This file and `PROTOCOL-COVERAGE.json` are the +copy this repository pins. Do not edit rows by hand: record a new manifest with ``` -node scripts/universe/generate-protocol-coverage.mjs --from +node scripts/universe/protocol-contract.mjs --record --from ``` -and verify with `node scripts/universe/generate-protocol-coverage.mjs --check`. +`node scripts/universe/protocol-contract.mjs --check` holds this repository's +own surfaces to the pinned roster, and +`node scripts/universe/protocol-contract.mjs --against ` fails when a +deployment serves a roster that differs from it. Release status semantics: every protocol starts BLOCKED and is upgraded only when its explorer integration is completed and verified against its Universe authority. -A protocol never silently disappears from this table. +A protocol never silently disappears from this table: `PROTOCOL-ROSTER.lock` +records every id that has been published, and the gate fails when one of them +stops appearing. -Registry version 1.0.0. 3 of 36 protocols are readable today; the rest are recorded here but not yet served. +Pinned from bitcoinuniverseio/backend-apis at commit 88a616cfdabc93facd3dcc3296f88dc214c24a40, +manifest schema universe-explorer-protocol-manifest-v1, registry version 1.0.0, +recorded 2026-08-30T18:18:38.168Z. + +6 of 38 protocols are readable today; the rest are recorded here but not yet served. | id | family | chain | authority | release status | coverage | |---|---|---|---|---|---| @@ -52,5 +62,7 @@ Registry version 1.0.0. 3 of 36 protocols are readable today; the rest are recor | doginals | OTHER | dogecoin | ord-dogecoin | BLOCKED | unknown | | drc20 | OTHER | dogecoin | ord-dogecoin | BLOCKED | unknown | | tap_doge | OTHER | dogecoin | index-doge-tap | BLOCKED | unknown | -| zerdinals | OTHER | zcash | index-zcash-metaprotocols | BLOCKED | unknown | -| zrunes | OTHER | zcash | index-zcash-metaprotocols | BLOCKED | unknown | +| dunes | OTHER | dogecoin | ord-dogecoin | BLOCKED | unknown | +| zerdinals | OTHER | zcash | index-zcash-metaprotocols | VERIFIED READ ONLY | complete | +| zrunes | OTHER | zcash | index-zcash-metaprotocols | VERIFIED READ ONLY | complete | +| zrc20 | OTHER | zcash | index-zcash-metaprotocols | VERIFIED READ ONLY | complete | diff --git a/docs/protocols/PROTOCOL-ROSTER.lock b/docs/protocols/PROTOCOL-ROSTER.lock new file mode 100644 index 0000000000..2ed72b2de7 --- /dev/null +++ b/docs/protocols/PROTOCOL-ROSTER.lock @@ -0,0 +1,41 @@ +# Every protocol id this explorer has published. +# The gate fails when one of these stops appearing in the pinned manifest, so a +# removal has to be a deliberate edit here rather than a quiet absence. +alkanes +arc20 +atomicals_nft +bitmap +block20 +brc20 +cat20 +chainbloom +dmt +doginals +drc20 +drops +dunes +dust20 +mezcal +names +op_drop +op_inscriptions +op_names +op_return +ordex +ordinals +patina +rare_sats +realms +runes +src101 +src20 +stamps +subrealms +tandem +tap +tap_doge +unat +witness_circles +zerdinals +zrc20 +zrunes diff --git a/scripts/universe/generate-protocol-coverage.mjs b/scripts/universe/generate-protocol-coverage.mjs deleted file mode 100644 index f96a27ae07..0000000000 --- a/scripts/universe/generate-protocol-coverage.mjs +++ /dev/null @@ -1,193 +0,0 @@ -#!/usr/bin/env node -/** - * Generates the protocol coverage documentation from the explorer protocol - * manifest. - * - * The manifest is whatever `/api/v1/universe/protocols` serves, which is the - * same contract the explorer frontend consumes. `docs/protocols/ - * PROTOCOL-COVERAGE.json` is the recorded manifest and `PROTOCOL-COVERAGE.md` - * is its human-readable table, so the table can never drift from the recorded - * manifest without `--check` failing. - * - * node scripts/universe/generate-protocol-coverage.mjs --from - * node scripts/universe/generate-protocol-coverage.mjs --check - * - * `--from` records a fresh manifest and rewrites both files. `--check` takes no - * source: it re-derives the table from the recorded manifest and fails if the - * committed table differs, which is the gate CI runs. - */ - -import { readFile, writeFile } from 'node:fs/promises'; -import { fileURLToPath } from 'node:url'; -import path from 'node:path'; - -const REPOSITORY_ROOT = path.resolve( - path.dirname(fileURLToPath(import.meta.url)), - '..', - '..', -); -const JSON_PATH = path.join( - REPOSITORY_ROOT, - 'docs', - 'protocols', - 'PROTOCOL-COVERAGE.json', -); -const MARKDOWN_PATH = path.join( - REPOSITORY_ROOT, - 'docs', - 'protocols', - 'PROTOCOL-COVERAGE.md', -); - -function usage(message) { - process.stderr.write(`${message}\n\n`); - process.stderr.write( - 'Usage:\n' + - ' generate-protocol-coverage.mjs --from record a manifest and rewrite the docs\n' + - ' generate-protocol-coverage.mjs --check verify the table matches the recorded manifest\n', - ); - process.exit(2); -} - -async function loadManifest(source) { - if (/^https?:\/\//i.test(source)) { - const response = await fetch(source, { - headers: { accept: 'application/json' }, - redirect: 'error', - signal: AbortSignal.timeout(30_000), - }); - if (!response.ok) { - throw new Error(`The manifest source answered HTTP ${response.status}.`); - } - return response.json(); - } - return JSON.parse(await readFile(path.resolve(source), 'utf8')); -} - -/** Rejects anything that is not a protocol manifest before it is recorded. */ -function validateManifest(manifest) { - if ( - typeof manifest !== 'object' || - manifest === null || - typeof manifest.registryVersion !== 'string' || - !Array.isArray(manifest.protocols) || - manifest.protocols.length === 0 - ) { - throw new Error( - 'The manifest must be an object with a registryVersion and a non-empty protocols array.', - ); - } - for (const protocol of manifest.protocols) { - for (const field of ['id', 'family', 'chain', 'releaseStatus']) { - if (typeof protocol?.[field] !== 'string' || !protocol[field]) { - throw new Error( - `Protocol ${protocol?.id ?? ''} is missing ${field}.`, - ); - } - } - } -} - -function coverageCell(coverage) { - if (coverage === null || coverage === undefined || coverage === '') { - return 'unknown'; - } - if (typeof coverage === 'string') return coverage; - return typeof coverage.state === 'string' ? coverage.state : 'unknown'; -} - -/** Escapes the pipe so an authority id can never break the table. */ -function cell(value) { - return String(value ?? '').replace(/\|/g, '\\|'); -} - -function renderMarkdown(manifest) { - const live = manifest.protocols.filter( - (protocol) => - protocol.releaseStatus === 'VERIFIED READ ONLY' || - protocol.releaseStatus === 'PRODUCTION VERIFIED', - ).length; - const rows = manifest.protocols.map( - (protocol) => - `| ${cell(protocol.id)} | ${cell(protocol.family)} | ${cell(protocol.chain)} | ` + - `${cell(protocol.indexerAuthority ?? 'none')} | ${cell(protocol.releaseStatus)} | ` + - `${cell(coverageCell(protocol.coverage))} |`, - ); - return `# Protocol coverage - -Generated from the explorer protocol registry in backend-apis -(src/universe-explorer/registry/explorer-protocol-registry.ts), as served by -\`/api/v1/universe/protocols\`. Do not edit rows by hand: regenerate with - -\`\`\` -node scripts/universe/generate-protocol-coverage.mjs --from -\`\`\` - -and verify with \`node scripts/universe/generate-protocol-coverage.mjs --check\`. - -Release status semantics: every protocol starts BLOCKED and is upgraded only when -its explorer integration is completed and verified against its Universe authority. -A protocol never silently disappears from this table. - -Registry version ${cell(manifest.registryVersion)}. ${live} of ${manifest.protocols.length} protocols are readable today; the rest are recorded here but not yet served. - -| id | family | chain | authority | release status | coverage | -|---|---|---|---|---|---| -${rows.join('\n')} -`; -} - -async function main() { - const argv = process.argv.slice(2); - const check = argv.includes('--check'); - const fromIndex = argv.indexOf('--from'); - const source = fromIndex === -1 ? null : argv[fromIndex + 1]; - - if (check && source) { - usage('Pass either --check or --from, not both.'); - } - if (!check && !source) { - usage('A manifest source is required.'); - } - - if (check) { - const manifest = JSON.parse(await readFile(JSON_PATH, 'utf8')); - validateManifest(manifest); - // Compare content, not line endings. This repository is developed on a - // Windows host where the working copy is checked out with CRLF, while the - // renderer emits LF, so a byte comparison reported every clean tree as - // stale and only ever passed on the Linux runner. - const normalise = (text) => text.split('\r\n').join('\n'); - const expected = normalise(renderMarkdown(manifest)); - const actual = normalise(await readFile(MARKDOWN_PATH, 'utf8')); - if (expected !== actual) { - process.stderr.write( - 'PROTOCOL-COVERAGE.md does not match PROTOCOL-COVERAGE.json.\n' + - 'Regenerate it with --from and commit the result.\n', - ); - process.exit(1); - } - process.stdout.write( - `Protocol coverage table matches the recorded manifest (${manifest.protocols.length} protocols).\n`, - ); - return; - } - - const manifest = await loadManifest(source); - validateManifest(manifest); - const recorded = { - registryVersion: manifest.registryVersion, - generatedAt: new Date().toISOString(), - protocols: manifest.protocols, - }; - await writeFile(JSON_PATH, `${JSON.stringify(recorded, null, 2)}\n`, 'utf8'); - await writeFile(MARKDOWN_PATH, renderMarkdown(recorded), 'utf8'); - process.stdout.write( - `Recorded ${recorded.protocols.length} protocols at registry version ${recorded.registryVersion}.\n`, - ); -} - -main().catch((error) => { - process.stderr.write(`${error instanceof Error ? error.message : error}\n`); - process.exit(1); -}); diff --git a/scripts/universe/protocol-contract.mjs b/scripts/universe/protocol-contract.mjs new file mode 100644 index 0000000000..81e58bb4b9 --- /dev/null +++ b/scripts/universe/protocol-contract.mjs @@ -0,0 +1,761 @@ +#!/usr/bin/env node +/** + * The protocol roster gate. + * + * The roster is owned by backend-apis: a TypeScript registry that + * `/api/v1/universe/protocols` serves and that + * `contracts/universe-explorer-protocols.json` records there. This repository + * consumes it, so it pins a copy and this gate holds the two together. + * + * What it replaced mattered. The old gate rendered a Markdown table from the + * recorded JSON and compared it against the committed Markdown, so the only + * thing it could prove was that a file agreed with itself. Two protocols, + * `dunes` and `zrc20`, were added to the registry and served in production + * while the recorded roster here still said thirty-six, and every run was + * green throughout. + * + * Three modes: + * + * --record --from pin a manifest and rewrite what is generated + * from it + * --check the offline gate CI runs on every commit + * --against compare the pinned manifest against what a + * deployment actually serves + * + * `--check` never reaches the network. It holds the pinned manifest to its + * schema and its provenance, holds this repository's own surfaces to the + * pinned roster, and refuses a roster that lost an entry. `--against` is the + * cross-repository half: it reads the served document and fails on any field + * of any protocol that differs. + */ + +import { readFile, writeFile } from 'node:fs/promises'; +import { fileURLToPath } from 'node:url'; +import path from 'node:path'; + +export const MANIFEST_SCHEMA_VERSION = 'universe-explorer-protocol-manifest-v1'; +export const PROTOCOL_SCHEMA_VERSION = 'universe-explorer-protocol-v1'; +export const SOURCE_REPOSITORY = 'bitcoinuniverseio/backend-apis'; + +const REPOSITORY_ROOT = path.resolve( + path.dirname(fileURLToPath(import.meta.url)), + '..', + '..', +); + +export const PATHS = { + manifest: path.join( + REPOSITORY_ROOT, + 'docs', + 'protocols', + 'PROTOCOL-COVERAGE.json', + ), + markdown: path.join( + REPOSITORY_ROOT, + 'docs', + 'protocols', + 'PROTOCOL-COVERAGE.md', + ), + roster: path.join( + REPOSITORY_ROOT, + 'docs', + 'protocols', + 'PROTOCOL-ROSTER.lock', + ), + readme: path.join(REPOSITORY_ROOT, 'README.md'), + protocolCopy: path.join( + REPOSITORY_ROOT, + 'frontend', + 'src', + 'app', + 'universe', + 'universe-protocol-copy.ts', + ), + apiService: path.join( + REPOSITORY_ROOT, + 'frontend', + 'src', + 'app', + 'universe', + 'universe-api.service.ts', + ), +}; + +const READABLE_STATUSES = new Set([ + 'VERIFIED READ ONLY', + 'PRODUCTION VERIFIED', +]); +const RELEASE_STATUSES = new Set([ + 'PRODUCTION VERIFIED', + 'VERIFIED READ ONLY', + 'BLOCKED', + 'INTENTIONALLY DISABLED', +]); +const COVERAGE_STATES = new Set([ + 'complete', + 'partial', + 'positive-only', + 'demand-populated', + 'unknown', +]); +const COMMIT_SHA = /^[0-9a-f]{7,64}$/; + +const README_MARKER_OPEN = ''; +const README_MARKER_CLOSE = ''; + +/** Compares content, not line endings: this tree is checked out with CRLF. */ +export const normalise = (text) => text.split('\r\n').join('\n'); + +export class GateFailure extends Error {} + +/** Collects every problem so one run names all of them, not just the first. */ +export class Report { + constructor() { + this.problems = []; + } + fail(message) { + this.problems.push(message); + } + throwIfFailed(headline) { + if (!this.problems.length) return; + throw new GateFailure( + `${headline}\n${this.problems.map((p) => ` - ${p}`).join('\n')}`, + ); + } +} + +// --------------------------------------------------------------------------- +// Manifest shape and provenance +// --------------------------------------------------------------------------- + +export function coverageState(coverage) { + if (coverage === null || coverage === undefined || coverage === '') { + return 'unknown'; + } + if (typeof coverage === 'string') return coverage; + return typeof coverage.state === 'string' ? coverage.state : 'unknown'; +} + +/** + * Refuses a roster that cannot be resolved to one entry. + * + * A protocol duplicated under incompatible ids, or an alias two entries both + * claim, makes every lookup answer by table order. The registry refuses to + * build one; this refuses to record one. + */ +export function assertRosterResolvesUniquely(protocols, report = new Report()) { + const owner = new Map(); + for (const protocol of protocols) { + const claims = [protocol?.id, ...(protocol?.aliases ?? [])]; + for (const claim of claims) { + if (typeof claim !== 'string' || !claim.trim()) { + report.fail( + `${protocol?.id ?? ''} claims an empty id or alias.`, + ); + continue; + } + const key = claim.trim().toLowerCase(); + const existing = owner.get(key); + if (existing && existing !== protocol.id) { + report.fail( + `"${key}" is claimed by both ${existing} and ${protocol.id}.`, + ); + } else if (existing === protocol.id && key !== protocol.id) { + report.fail(`${protocol.id} repeats the alias "${key}".`); + } + owner.set(key, protocol.id); + } + } + return report; +} + +/** + * Holds a manifest to its envelope. + * + * A roster with no provenance can only be compared against whatever is being + * served right now, which is a check that passes the day it is written and + * says nothing after. Every field here exists so a mismatch names both sides. + */ +export function validateManifest(manifest, report = new Report()) { + if (typeof manifest !== 'object' || manifest === null) { + report.fail('The manifest is not an object.'); + return report; + } + if (manifest.schemaVersion !== MANIFEST_SCHEMA_VERSION) { + report.fail( + `The manifest schema is ${JSON.stringify(manifest.schemaVersion)}; this gate reads ${MANIFEST_SCHEMA_VERSION}.`, + ); + } + if ( + typeof manifest.registryVersion !== 'string' || + !manifest.registryVersion + ) { + report.fail('The manifest carries no registryVersion.'); + } + if (manifest.sourceRepository !== SOURCE_REPOSITORY) { + report.fail( + `The manifest names ${JSON.stringify(manifest.sourceRepository)} as its source; the roster is owned by ${SOURCE_REPOSITORY}.`, + ); + } + if ( + typeof manifest.sourceSha !== 'string' || + !COMMIT_SHA.test(manifest.sourceSha) + ) { + report.fail( + `The manifest names no commit it was produced from (sourceSha ${JSON.stringify(manifest.sourceSha)}).`, + ); + } + if ( + typeof manifest.generatedAt !== 'string' || + Number.isNaN(Date.parse(manifest.generatedAt)) + ) { + report.fail('The manifest carries no readable generatedAt time.'); + } + if (!Array.isArray(manifest.primaryStrip) || !manifest.primaryStrip.length) { + report.fail('The manifest carries no primaryStrip.'); + } + if (!Array.isArray(manifest.protocols) || !manifest.protocols.length) { + report.fail('The manifest carries no protocols.'); + return report; + } + + for (const protocol of manifest.protocols) { + const id = protocol?.id ?? ''; + if (protocol?.schemaVersion !== PROTOCOL_SCHEMA_VERSION) { + report.fail( + `${id} carries schema ${JSON.stringify(protocol?.schemaVersion)}.`, + ); + } + for (const field of ['id', 'displayName', 'shortName', 'family', 'chain']) { + if (typeof protocol?.[field] !== 'string' || !protocol[field]) { + report.fail(`${id} is missing ${field}.`); + } + } + for (const field of [ + 'aliases', + 'networks', + 'implementedReadOperations', + 'authorizedReadOperations', + ]) { + if (!Array.isArray(protocol?.[field])) { + report.fail(`${id} is missing the ${field} list.`); + } + } + if (!RELEASE_STATUSES.has(protocol?.releaseStatus)) { + report.fail( + `${id} has release status ${JSON.stringify(protocol?.releaseStatus)}.`, + ); + } + if (!COVERAGE_STATES.has(coverageState(protocol?.coverage))) { + report.fail(`${id} has coverage ${JSON.stringify(protocol?.coverage)}.`); + } + if ( + READABLE_STATUSES.has(protocol?.releaseStatus) && + (typeof protocol?.indexerAuthority !== 'string' || + !protocol.indexerAuthority) + ) { + report.fail(`${id} is marked readable but names no authority.`); + } + } + + assertRosterResolvesUniquely(manifest.protocols, report); + return report; +} + +/** Every id and alias the roster answers to, lowercased. */ +export function resolvableNames(protocols) { + const names = new Map(); + for (const protocol of protocols) { + names.set(protocol.id.toLowerCase(), protocol); + for (const alias of protocol.aliases ?? []) { + names.set(String(alias).toLowerCase(), protocol); + } + } + return names; +} + +// --------------------------------------------------------------------------- +// Generated documents +// --------------------------------------------------------------------------- + +/** Escapes the pipe so an authority id can never break the table. */ +function cell(value) { + return String(value ?? '').replace(/\|/g, '\\|'); +} + +export function readableProtocols(manifest) { + return manifest.protocols.filter((p) => + READABLE_STATUSES.has(p.releaseStatus), + ); +} + +export function renderMarkdown(manifest) { + const readable = readableProtocols(manifest); + const rows = manifest.protocols.map( + (protocol) => + `| ${cell(protocol.id)} | ${cell(protocol.family)} | ${cell(protocol.chain)} | ` + + `${cell(protocol.indexerAuthority ?? 'none')} | ${cell(protocol.releaseStatus)} | ` + + `${cell(coverageState(protocol.coverage))} |`, + ); + return `# Protocol coverage + +The roster is owned by \`${cell(manifest.sourceRepository)}\`, in +\`src/universe-explorer/registry/explorer-protocol-registry.ts\`, and served by +\`/api/v1/universe/protocols\`. This file and \`PROTOCOL-COVERAGE.json\` are the +copy this repository pins. Do not edit rows by hand: record a new manifest with + +\`\`\` +node scripts/universe/protocol-contract.mjs --record --from +\`\`\` + +\`node scripts/universe/protocol-contract.mjs --check\` holds this repository's +own surfaces to the pinned roster, and +\`node scripts/universe/protocol-contract.mjs --against \` fails when a +deployment serves a roster that differs from it. + +Release status semantics: every protocol starts BLOCKED and is upgraded only when +its explorer integration is completed and verified against its Universe authority. +A protocol never silently disappears from this table: \`PROTOCOL-ROSTER.lock\` +records every id that has been published, and the gate fails when one of them +stops appearing. + +Pinned from ${cell(manifest.sourceRepository)} at commit ${cell(manifest.sourceSha)}, +manifest schema ${cell(manifest.schemaVersion)}, registry version ${cell(manifest.registryVersion)}, +recorded ${cell(manifest.generatedAt)}. + +${readable.length} of ${manifest.protocols.length} protocols are readable today; the rest are recorded here but not yet served. + +| id | family | chain | authority | release status | coverage | +|---|---|---|---|---|---| +${rows.join('\n')} +`; +} + +/** + * The readable list the README publishes. + * + * The README said three protocols were readable, backed by Ord, for as long + * as three Zcash protocols had also been readable. Generating the sentence + * from the pinned roster is the only version of it that stays true. + */ +export function renderReadmeBlock(manifest) { + const readable = readableProtocols(manifest); + const byChain = new Map(); + for (const protocol of readable) { + if (!byChain.has(protocol.chain)) byChain.set(protocol.chain, []); + byChain.get(protocol.chain).push(protocol); + } + const lines = [...byChain.entries()].map(([chain, protocols]) => { + const authorities = [ + ...new Set(protocols.map((p) => p.indexerAuthority).filter(Boolean)), + ]; + const names = protocols.map((p) => `**${p.displayName}**`).join(', '); + return `- On ${chain}: ${names}, from ${authorities.join(' and ')}.`; + }); + return [ + README_MARKER_OPEN, + '', + `${readable.length} of the ${manifest.protocols.length} protocols in the registry are readable today:`, + '', + ...lines, + '', + README_MARKER_CLOSE, + ].join('\n'); +} + +export function replaceReadmeBlock(readme, block) { + const text = normalise(readme); + const start = text.indexOf(README_MARKER_OPEN); + const end = text.indexOf(README_MARKER_CLOSE); + if (start === -1 || end === -1 || end < start) { + throw new GateFailure( + `README.md carries no ${README_MARKER_OPEN} block for the readable protocol list.`, + ); + } + return ( + text.slice(0, start) + block + text.slice(end + README_MARKER_CLOSE.length) + ); +} + +export function readmeBlockOf(readme) { + const text = normalise(readme); + const start = text.indexOf(README_MARKER_OPEN); + const end = text.indexOf(README_MARKER_CLOSE); + if (start === -1 || end === -1 || end < start) return null; + return text.slice(start, end + README_MARKER_CLOSE.length); +} + +// --------------------------------------------------------------------------- +// This repository's own surfaces +// --------------------------------------------------------------------------- + +/** The protocol ids `universe-protocol-copy.ts` writes prose for. */ +export function protocolCopyIds(source) { + const start = source.indexOf('const PROTOCOL_COPY'); + if (start === -1) { + throw new GateFailure( + 'universe-protocol-copy.ts no longer declares PROTOCOL_COPY; the gate cannot read its ids.', + ); + } + return [...source.slice(start).matchAll(/^ {2}([a-z0-9_]+): \{/gm)].map( + (match) => match[1], + ); +} + +/** The chain protocol paths the API client will call, by chain. */ +export function apiAllowlist(source) { + const match = source.match( + /const allowed = chain === '(\w+)'\s*\?\s*\[([^\]]*)\]\s*:\s*\[([^\]]*)\];/, + ); + if (!match) { + throw new GateFailure( + 'universe-api.service.ts no longer declares the chain protocol allowlist in a shape the gate can read.', + ); + } + const parse = (list) => + [...list.matchAll(/'([^']+)'/g)].map((entry) => entry[1]); + const namedChain = match[1]; + const otherChain = namedChain === 'dogecoin' ? 'zcash' : 'dogecoin'; + return new Map([ + [namedChain, parse(match[2])], + [otherChain, parse(match[3])], + ]); +} + +/** + * Holds this repository's surfaces to the pinned roster. + * + * Both directions matter. A surface naming a protocol the registry does not + * carry is a route to something the product does not have; a readable + * protocol with no route is a capability the product hides. + */ +export function checkSurfaces(manifest, sources, report = new Report()) { + const names = resolvableNames(manifest.protocols); + + for (const id of protocolCopyIds(sources.protocolCopy)) { + if (!names.has(id.toLowerCase())) { + report.fail( + `universe-protocol-copy.ts writes prose for "${id}", which is not in the registry.`, + ); + } + } + + const allowlist = apiAllowlist(sources.apiService); + for (const [chain, entries] of allowlist) { + for (const entry of entries) { + const protocol = names.get(entry.toLowerCase()); + if (!protocol) { + report.fail( + `the ${chain} API allowlist calls "${entry}", which is not in the registry.`, + ); + continue; + } + if (protocol.chain !== chain) { + report.fail( + `the ${chain} API allowlist calls "${entry}", which the registry places on ${protocol.chain}.`, + ); + } + } + const routed = new Set( + entries.map((entry) => names.get(entry.toLowerCase())?.id).filter(Boolean), + ); + for (const protocol of manifest.protocols) { + if (protocol.chain !== chain) continue; + if (!READABLE_STATUSES.has(protocol.releaseStatus)) continue; + if (!routed.has(protocol.id)) { + report.fail( + `${protocol.id} is readable on ${chain} but the API allowlist has no path for it.`, + ); + } + } + } + + return report; +} + +/** + * Refuses a roster that quietly lost an entry. + * + * The lock is every id that has been published. Removing a protocol is a + * decision, and a decision leaves a diff: the id has to come out of the lock + * in the same commit that takes it out of the registry. + */ +export function checkRoster(manifest, lock, report = new Report()) { + const locked = lock + .split('\n') + .map((line) => line.replace(/#.*$/, '').trim()) + .filter(Boolean); + const present = new Set(manifest.protocols.map((p) => p.id)); + for (const id of locked) { + if (!present.has(id)) { + report.fail( + `${id} is in PROTOCOL-ROSTER.lock but no longer in the pinned manifest. A protocol may not disappear silently: take it out of the lock in the same commit if that is intended.`, + ); + } + } + const lockedSet = new Set(locked); + for (const id of present) { + if (!lockedSet.has(id)) { + report.fail( + `${id} is in the pinned manifest but not in PROTOCOL-ROSTER.lock. Record it with --record.`, + ); + } + } + return report; +} + +export function renderRoster(manifest) { + return `# Every protocol id this explorer has published. +# The gate fails when one of these stops appearing in the pinned manifest, so a +# removal has to be a deliberate edit here rather than a quiet absence. +${[...manifest.protocols.map((p) => p.id)].sort().join('\n')} +`; +} + +// --------------------------------------------------------------------------- +// Cross-repository comparison +// --------------------------------------------------------------------------- + +/** The part of a document that is a claim about the roster. */ +export function comparable(document) { + return { + schemaVersion: document.schemaVersion, + registryVersion: document.registryVersion, + sourceRepository: document.sourceRepository, + primaryStrip: document.primaryStrip, + protocols: document.protocols, + }; +} + +/** + * Names every difference between the pinned roster and a served one. + * + * Reporting the whole diff rather than the first mismatch is the difference + * between "the roster moved" and a list of what moved, which is what someone + * reading a failed release needs. + */ +export function diffManifests(pinned, served, report = new Report()) { + for (const field of [ + 'schemaVersion', + 'registryVersion', + 'sourceRepository', + ]) { + if (pinned[field] !== served[field]) { + report.fail( + `${field}: pinned ${JSON.stringify(pinned[field])}, served ${JSON.stringify(served[field])}.`, + ); + } + } + if ( + JSON.stringify(pinned.primaryStrip) !== JSON.stringify(served.primaryStrip) + ) { + report.fail( + `primaryStrip: pinned ${JSON.stringify(pinned.primaryStrip)}, served ${JSON.stringify(served.primaryStrip)}.`, + ); + } + + const pinnedById = new Map((pinned.protocols ?? []).map((p) => [p.id, p])); + const servedById = new Map((served.protocols ?? []).map((p) => [p.id, p])); + + for (const id of servedById.keys()) { + if (!pinnedById.has(id)) { + report.fail(`${id} is served but is not in the pinned manifest.`); + } + } + for (const id of pinnedById.keys()) { + if (!servedById.has(id)) { + report.fail(`${id} is pinned but is not served.`); + } + } + for (const [id, pinnedEntry] of pinnedById) { + const servedEntry = servedById.get(id); + if (!servedEntry) continue; + const fields = new Set([ + ...Object.keys(pinnedEntry), + ...Object.keys(servedEntry), + ]); + for (const field of fields) { + const a = JSON.stringify(pinnedEntry[field]); + const b = JSON.stringify(servedEntry[field]); + if (a !== b) { + report.fail(`${id}.${field}: pinned ${a}, served ${b}.`); + } + } + } + + const order = (document) => (document.protocols ?? []).map((p) => p.id); + if ( + report.problems.length === 0 && + JSON.stringify(order(pinned)) !== JSON.stringify(order(served)) + ) { + report.fail( + 'the roster order differs, which changes the order of the public protocol strip.', + ); + } + return report; +} + +// --------------------------------------------------------------------------- +// Entry points +// --------------------------------------------------------------------------- + +async function loadDocument(source) { + if (/^https?:\/\//i.test(source)) { + const url = /\/api\//.test(source) + ? source + : `${source.replace(/\/+$/, '')}/api/v1/universe/protocols`; + const response = await fetch(url, { + headers: { accept: 'application/json' }, + redirect: 'error', + signal: AbortSignal.timeout(30_000), + }); + if (!response.ok) { + throw new GateFailure( + `${url} answered HTTP ${response.status}; the roster could not be read.`, + ); + } + return response.json(); + } + return JSON.parse(await readFile(path.resolve(source), 'utf8')); +} + +async function readSources() { + const [protocolCopy, apiService, markdown, roster, readme] = + await Promise.all([ + readFile(PATHS.protocolCopy, 'utf8'), + readFile(PATHS.apiService, 'utf8'), + readFile(PATHS.markdown, 'utf8'), + readFile(PATHS.roster, 'utf8'), + readFile(PATHS.readme, 'utf8'), + ]); + return { protocolCopy, apiService, markdown, roster, readme }; +} + +async function record(source) { + const document = await loadDocument(source); + validateManifest(document).throwIfFailed( + 'The manifest that was read cannot be recorded.', + ); + const readme = await readFile(PATHS.readme, 'utf8'); + await writeFile( + PATHS.manifest, + `${JSON.stringify(document, null, 2)}\n`, + 'utf8', + ); + await writeFile(PATHS.markdown, renderMarkdown(document), 'utf8'); + await writeFile(PATHS.roster, renderRoster(document), 'utf8'); + await writeFile( + PATHS.readme, + replaceReadmeBlock(readme, renderReadmeBlock(document)), + 'utf8', + ); + process.stdout.write( + `Pinned ${document.protocols.length} protocols from ${document.sourceRepository} at ${document.sourceSha}.\n`, + ); +} + +async function check() { + const manifest = JSON.parse(await readFile(PATHS.manifest, 'utf8')); + validateManifest(manifest).throwIfFailed( + 'The pinned protocol manifest is not a manifest this gate can trust.', + ); + const sources = await readSources(); + + const report = new Report(); + if (normalise(renderMarkdown(manifest)) !== normalise(sources.markdown)) { + report.fail( + 'PROTOCOL-COVERAGE.md no longer matches the pinned manifest. Regenerate with --record.', + ); + } + const readmeBlock = readmeBlockOf(sources.readme); + if (readmeBlock === null) { + report.fail( + `README.md carries no ${README_MARKER_OPEN} block for the readable protocol list.`, + ); + } else if (readmeBlock !== renderReadmeBlock(manifest)) { + report.fail( + 'The README readable-protocol list no longer matches the pinned manifest. Regenerate with --record.', + ); + } + checkRoster(manifest, sources.roster, report); + checkSurfaces(manifest, sources, report); + report.throwIfFailed('The protocol roster gate failed.'); + + process.stdout.write( + `The pinned roster holds: ${manifest.protocols.length} protocols from ` + + `${manifest.sourceRepository} at ${manifest.sourceSha}, ` + + `${readableProtocols(manifest).length} readable, every surface accounted for.\n`, + ); +} + +async function against(source) { + const [pinned, served] = await Promise.all([ + readFile(PATHS.manifest, 'utf8').then(JSON.parse), + loadDocument(source), + ]); + validateManifest(served).throwIfFailed( + `${source} did not serve a manifest this gate can read.`, + ); + const report = diffManifests(comparable(pinned), comparable(served)); + if (report.problems.length) { + report.problems.push( + `pinned from ${pinned.sourceSha}, served by ${served.sourceSha}. ` + + 'Record the served roster with --record once the deployment is the one this release intends.', + ); + } + report.throwIfFailed( + 'What is served does not match the pinned protocol roster.', + ); + process.stdout.write( + `The served roster matches the pinned one: ${served.protocols.length} protocols, ` + + `registry ${served.registryVersion}, served by ${served.sourceSha}.\n`, + ); +} + +function usage(message) { + process.stderr.write(`${message}\n\n`); + process.stderr.write( + 'Usage:\n' + + ' protocol-contract.mjs --record --from pin a manifest and rewrite what it generates\n' + + ' protocol-contract.mjs --check the offline gate\n' + + ' protocol-contract.mjs --against compare the pin against what is served\n', + ); + process.exit(2); +} + +async function main() { + const argv = process.argv.slice(2); + const fromIndex = argv.indexOf('--from'); + const againstIndex = argv.indexOf('--against'); + const wantsRecord = argv.includes('--record'); + const wantsCheck = argv.includes('--check'); + + const modes = [wantsRecord, wantsCheck, againstIndex !== -1].filter(Boolean); + if (modes.length !== 1) { + usage('Pass exactly one of --record, --check, --against.'); + } + if (wantsRecord) { + if (fromIndex === -1 || !argv[fromIndex + 1]) { + usage('--record needs --from .'); + } + await record(argv[fromIndex + 1]); + return; + } + if (wantsCheck) { + await check(); + return; + } + if (!argv[againstIndex + 1]) usage('--against needs a url or file.'); + await against(argv[againstIndex + 1]); +} + +const invokedDirectly = + process.argv[1] && + path.resolve(process.argv[1]) === fileURLToPath(import.meta.url); + +if (invokedDirectly) { + main().catch((error) => { + process.stderr.write(`${error instanceof Error ? error.message : error}\n`); + process.exit(1); + }); +} diff --git a/scripts/universe/protocol-contract.test.mjs b/scripts/universe/protocol-contract.test.mjs new file mode 100644 index 0000000000..4c055c627f --- /dev/null +++ b/scripts/universe/protocol-contract.test.mjs @@ -0,0 +1,267 @@ +import assert from 'node:assert/strict'; +import { readFile } from 'node:fs/promises'; +import test from 'node:test'; + +import { + PATHS, + apiAllowlist, + assertRosterResolvesUniquely, + checkRoster, + checkSurfaces, + comparable, + diffManifests, + protocolCopyIds, + readmeBlockOf, + renderReadmeBlock, + renderRoster, + validateManifest, +} from './protocol-contract.mjs'; + +/** + * A gate that has only ever passed has not shown it can fail. + * + * The gate this replaced was green while production served two protocols the + * recorded roster did not carry, so every case below is one the old gate could + * not see, written against the roster this repository actually pins. + */ + +const pinned = JSON.parse(await readFile(PATHS.manifest, 'utf8')); + +/** A copy with one thing changed, so a case says exactly what it varied. */ +function withProtocol(manifest, id, changes) { + return { + ...manifest, + protocols: manifest.protocols.map((protocol) => + protocol.id === id ? { ...protocol, ...changes } : protocol, + ), + }; +} + +function problems(report) { + return report.problems.join('\n'); +} + +test('the roster this repository pins is one the gate accepts', () => { + assert.deepEqual(validateManifest(pinned).problems, []); +}); + +test('a manifest with no provenance is refused', () => { + const stripped = { ...pinned }; + delete stripped.sourceSha; + delete stripped.sourceRepository; + const found = problems(validateManifest(stripped)); + assert.match(found, /names no commit it was produced from/); + assert.match(found, /the roster is owned by bitcoinuniverseio\/backend-apis/); +}); + +test('a manifest from another schema is refused', () => { + const found = problems( + validateManifest({ ...pinned, schemaVersion: 'something-else' }), + ); + assert.match(found, /this gate reads universe-explorer-protocol-manifest-v1/); +}); + +test('a protocol that claims to be readable with no authority is refused', () => { + const found = problems( + validateManifest( + withProtocol(pinned, 'dunes', { + releaseStatus: 'VERIFIED READ ONLY', + indexerAuthority: undefined, + }), + ), + ); + assert.match(found, /dunes is marked readable but names no authority/); +}); + +test('a protocol duplicated under incompatible ids is refused', () => { + const doubled = { + ...pinned, + protocols: [ + ...pinned.protocols, + { ...pinned.protocols[0], id: 'runes_v2', aliases: ['runes'] }, + ], + }; + const found = problems(assertRosterResolvesUniquely(doubled.protocols)); + assert.match(found, /"runes" is claimed by both runes and runes_v2/); +}); + +test('an alias two protocols both claim is refused', () => { + const clashing = [ + { id: 'a', aliases: ['shared'] }, + { id: 'b', aliases: ['shared'] }, + ]; + assert.match( + problems(assertRosterResolvesUniquely(clashing)), + /"shared" is claimed by both a and b/, + ); +}); + +test('a protocol that disappears from the roster fails the lock', () => { + const lock = renderRoster(pinned); + const shrunk = { + ...pinned, + protocols: pinned.protocols.filter((p) => p.id !== 'zrc20'), + }; + assert.match( + problems(checkRoster(shrunk, lock)), + /zrc20 is in PROTOCOL-ROSTER\.lock but no longer in the pinned manifest/, + ); +}); + +test('a protocol added without recording it fails the lock', () => { + const lock = renderRoster({ + ...pinned, + protocols: pinned.protocols.filter((p) => p.id !== 'dunes'), + }); + assert.match( + problems(checkRoster(pinned, lock)), + /dunes is in the pinned manifest but not in PROTOCOL-ROSTER\.lock/, + ); +}); + +test('the lock accepts the roster it was rendered from', () => { + assert.deepEqual(checkRoster(pinned, renderRoster(pinned)).problems, []); +}); + +test('this repository names no protocol the registry does not carry', async () => { + const sources = { + protocolCopy: await readFile(PATHS.protocolCopy, 'utf8'), + apiService: await readFile(PATHS.apiService, 'utf8'), + }; + assert.deepEqual(checkSurfaces(pinned, sources).problems, []); +}); + +test('prose for a protocol the registry dropped is a failure', async () => { + const sources = { + protocolCopy: `const PROTOCOL_COPY = {\n not_a_protocol: {\n`, + apiService: await readFile(PATHS.apiService, 'utf8'), + }; + assert.match( + problems(checkSurfaces(pinned, sources)), + /writes prose for "not_a_protocol", which is not in the registry/, + ); +}); + +test('an API path for a protocol the registry does not carry is a failure', async () => { + const sources = { + protocolCopy: await readFile(PATHS.protocolCopy, 'utf8'), + apiService: + "const allowed = chain === 'dogecoin'\n" + + " ? ['doginals', 'drc20', 'doge-tap', 'dunes', 'invented']\n" + + " : ['zerdinals', 'zrunes', 'zrc20'];", + }; + assert.match( + problems(checkSurfaces(pinned, sources)), + /the dogecoin API allowlist calls "invented", which is not in the registry/, + ); +}); + +test('an API path pointed at the wrong chain is a failure', async () => { + const sources = { + protocolCopy: await readFile(PATHS.protocolCopy, 'utf8'), + apiService: + "const allowed = chain === 'dogecoin'\n" + + " ? ['doginals', 'drc20', 'doge-tap', 'dunes', 'zrunes']\n" + + " : ['zerdinals', 'zrunes', 'zrc20'];", + }; + assert.match( + problems(checkSurfaces(pinned, sources)), + /the dogecoin API allowlist calls "zrunes", which the registry places on zcash/, + ); +}); + +test('a readable protocol with no route is a failure', async () => { + const sources = { + protocolCopy: await readFile(PATHS.protocolCopy, 'utf8'), + apiService: + "const allowed = chain === 'dogecoin'\n" + + " ? ['doginals', 'drc20', 'doge-tap', 'dunes']\n" + + " : ['zerdinals', 'zrunes'];", + }; + assert.match( + problems(checkSurfaces(pinned, sources)), + /zrc20 is readable on zcash but the API allowlist has no path for it/, + ); +}); + +test('the allowlist reader follows the aliases the registry publishes', async () => { + const allowlist = apiAllowlist(await readFile(PATHS.apiService, 'utf8')); + assert.ok(allowlist.get('dogecoin').includes('doge-tap')); + const tapDoge = pinned.protocols.find((p) => p.id === 'tap_doge'); + assert.ok(tapDoge.aliases.includes('doge-tap')); +}); + +test('the copy reader finds the ids the frontend actually writes prose for', async () => { + const ids = protocolCopyIds(await readFile(PATHS.protocolCopy, 'utf8')); + assert.ok(ids.includes('ordinals')); + assert.ok(ids.includes('drc20')); +}); + +test('the README block states the roster it was generated from', async () => { + const block = readmeBlockOf(await readFile(PATHS.readme, 'utf8')); + assert.equal(block, renderReadmeBlock(pinned)); + assert.match(block, /6 of the 38 protocols in the registry are readable/); +}); + +test('a served roster that gained a protocol is named, not tolerated', () => { + const served = { + ...pinned, + protocols: [ + ...pinned.protocols, + { ...pinned.protocols[0], id: 'brand_new', aliases: [] }, + ], + }; + assert.match( + problems(diffManifests(comparable(pinned), comparable(served))), + /brand_new is served but is not in the pinned manifest/, + ); +}); + +test('a served roster that dropped a protocol is named', () => { + const served = { + ...pinned, + protocols: pinned.protocols.filter((p) => p.id !== 'dunes'), + }; + assert.match( + problems(diffManifests(comparable(pinned), comparable(served))), + /dunes is pinned but is not served/, + ); +}); + +test('a served roster that changed one field of one protocol is named', () => { + const served = withProtocol(pinned, 'doginals', { + releaseStatus: 'VERIFIED READ ONLY', + }); + assert.match( + problems(diffManifests(comparable(pinned), comparable(served))), + /doginals\.releaseStatus: pinned "BLOCKED", served "VERIFIED READ ONLY"/, + ); +}); + +test('a served roster that changed an authority is named', () => { + const served = withProtocol(pinned, 'zrunes', { + indexerAuthority: 'somebody-elses-indexer', + }); + assert.match( + problems(diffManifests(comparable(pinned), comparable(served))), + /zrunes\.indexerAuthority: pinned "index-zcash-metaprotocols", served "somebody-elses-indexer"/, + ); +}); + +test('a served roster in a different order is named', () => { + const served = { + ...pinned, + protocols: [...pinned.protocols].reverse(), + }; + assert.match( + problems(diffManifests(comparable(pinned), comparable(served))), + /the roster order differs/, + ); +}); + +test('a served roster identical to the pin reports nothing', () => { + assert.deepEqual( + diffManifests(comparable(pinned), comparable(pinned)).problems, + [], + ); +}); From e408feb207c0ec810ca7cf5a18efb99af3ab4ae0 Mon Sep 17 00:00:00 2001 From: Bitcoin Universe Date: Sun, 30 Aug 2026 18:33:47 +0000 Subject: [PATCH 06/15] Split the deep visual pass so the gate finishes The comment in this job predicted this failure and named the fix. The deep pass is routes times states times themes times widths, it was measured at 47 minutes over seven routes on 2026-08-29, and it was measured again today at 1h11m on run 33323262662. The job cap is 100 minutes. On develop it reached the cap: run 33320958080 was killed at 1h41m with the deep pass still running, so the required check for the promotion pull request never produced a verdict at all, and the same happened to its pull request run beside it. Three shards, split by route because the cost is per route, sized from the measurement rather than by counting names: graphs and mining, then protocols and home, then blocks, transactions and addresses with the chain failure states. About 20, 20 and 34 minutes of a 60 minute cap. Each builds its own copy, which is what the mobile gate beside it already does and for the same reason: the gates cannot make each other wait, and each one can be read and run on its own. Nothing is dropped. Every route, state, theme and width the pass covered is still covered, and the broad pass, the forced-colour pass and the bundle gates stay in the build job, which now fits in 45 minutes. Co-Authored-By: Claude Opus 5 --- .github/workflows/universe-ci.yml | 178 ++++++++++++++++++++---------- 1 file changed, 121 insertions(+), 57 deletions(-) diff --git a/.github/workflows/universe-ci.yml b/.github/workflows/universe-ci.yml index cc2f76318c..b35581b4d0 100644 --- a/.github/workflows/universe-ci.yml +++ b/.github/workflows/universe-ci.yml @@ -134,12 +134,12 @@ jobs: frontend: name: Frontend build runs-on: [self-hosted, linux-ultra] - # The deep visual pass measured 48 minutes once the gated route list grew - # from three routes to seven, which put the job over the hour it had and - # killed it three minutes from the end. A gate that cannot finish protects - # nothing, which is the same reason the matrix was split in two to begin - # with. Budget is for the pass that actually runs, with room to grow. - timeout-minutes: 100 + # This builds, gates the bundle, and walks every route once at a phone and + # a desktop width. The deep pass that used to follow it now runs beside it, + # sharded, because it stopped fitting: 71 minutes of a 100 minute cap on + # its own, and on develop it reached the cap and the required check died as + # a timeout. Budget here is for the work that is left, with room to grow. + timeout-minutes: 45 steps: - name: Checkout uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 @@ -224,64 +224,133 @@ jobs: # chart panel draws nothing, or when a failure state says nothing about # why. Those three states passed every other gate here and shipped. # - # Split into two passes that finish, rather than one sweep that does not. - # Every combination of thirteen routes, seven data states, three themes - # and seven widths is nineteen hundred page loads, which does not fit the - # job timeout, and a gate that cannot finish protects nothing. - # - # The broad pass walks every route at a phone and a desktop width with - # real data, which is where overflow, broken images and a page that never - # finishes show up. The deep pass takes the routes this suite holds to a - # terminal state through every failure state, every theme and the widths - # that change the layout, which is where a page that lies about why it is - # empty shows up. - # - # Measured on run 33254326169, 2026-08-29, on universe-linux-ultra-02: - # - # broad pass 64 screenshots 5m50s - # deep pass 504 screenshots 47m10s - # chain failures 48 screenshots 3m27s - # forced colours and zoom 30s - # whole job 59m57s of a 100 minute cap + # The pass here is the broad one: every route, real data, a phone and a + # desktop width, which is where overflow, broken images and a page that + # never finishes show up. It measured 5m50s. # - # This comment said two and a half minutes and sixteen. The deep pass has - # since grown to roughly three times that, it is now 78 percent of the - # job, and the job sits at 60 of its 100 minutes. Nothing is failing. The - # point of writing the real numbers down is that the next person to add a - # route, a state or a theme to the deep pass is the one who meets the - # cap, and without this they meet it as a timeout forty-seven minutes - # into a run rather than as a number they could have added up first. - # - # The deep pass is routes multiplied by states multiplied by themes - # multiplied by widths. Adding one route to it currently costs about - # seven minutes. If it needs to grow again, split it the way it was split - # from the broad pass in the first place, rather than raising the cap: - # a gate that cannot finish protects nothing, which is the reason the - # split exists. + # The deep pass is the `visual` job below. It is routes multiplied by + # states multiplied by themes multiplied by widths, it grew to 71 minutes + # of this job's 100 minute cap, and on develop it reached the cap: run + # 33320958080 was killed at 1h41 with the deep pass still going, so the + # required check for the promotion had no verdict at all. The comment + # that used to sit here said to split it again rather than raise the cap, + # because a gate that cannot finish protects nothing. That is what the + # shards are. - name: Every route renders and finishes, at a phone and a desktop width working-directory: scripts/universe/visual-qa run: >- node capture.mjs --base=http://127.0.0.1:$GATEWAY_PORT --out=./artifacts-broad --states=populated --themes=default --viewports=375,1440 + - name: Forced colours and 200 percent zoom + working-directory: scripts/universe/visual-qa + run: node modes-check.mjs --base=http://127.0.0.1:$GATEWAY_PORT + + - name: Keep the contrast report and the route screenshots + if: always() + uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 + with: + name: universe-visual-qa-broad + path: scripts/universe/visual-qa/artifacts-broad + retention-days: 30 + + # The deep visual pass, in three shards. + # + # Measured on run 33323262662, 2026-08-30, on universe-linux-ultra-04: seven + # gated routes across three widths, seven states and the themes they carry + # took 1h11m in one step. Ten minutes a route. One more route would have put + # the job past its cap; on develop it already had. + # + # The split is by route because the cost is per route, and the shards are + # sized from that measurement rather than by counting names. Each shard + # builds its own copy, the way the mobile gate does: it costs one production + # build and buys a gate that can be read, run and debugged on its own, and + # that cannot make the job beside it wait. + # + # charts-and-mining graphs, mining about 20 minutes + # protocols-and-home protocols, home about 20 minutes + # blocks-and-transactions blocks, tx, address about 30 minutes + # plus the chain failure states, 3m27s + visual: + name: Visual matrix (${{ matrix.shard }}) + runs-on: [self-hosted, linux-ultra] + timeout-minutes: 60 + strategy: + # One shard failing says nothing about the others, and finding out that + # two route groups are broken in one run beats finding out twice. + fail-fast: false + matrix: + include: + - shard: charts-and-mining + routes: graphs,mining + chainStates: 'no' + - shard: protocols-and-home + routes: protocols,home + chainStates: 'no' + - shard: blocks-and-transactions + routes: blocks,tx,address + chainStates: 'yes' + steps: + - name: Checkout + uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + + - name: Setup Node + uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4 + with: + node-version: '24.19.0' + cache: npm + cache-dependency-path: frontend/package-lock.json + + - name: Install + working-directory: frontend + run: npm ci + + - name: Production build + working-directory: frontend + run: npm run build:universe + + - name: Install the visual QA harness + working-directory: scripts/universe/visual-qa + run: npm ci + + # A fixed port here once measured another job's build: two runners share + # a host, the second gateway crashed on the taken port behind "&", the + # probe happily reached the first job's server, and the run died halfway + # when that other job ended. Each run now serves on its own free port and + # refuses to continue unless the process answering is the one it started. + - name: Serve the built frontend + run: | + GATEWAY_PORT=$(node -e "const s=require('net').createServer();s.listen(0,'127.0.0.1',()=>{console.log(s.address().port);s.close()})") + echo "GATEWAY_PORT=$GATEWAY_PORT" >> "$GITHUB_ENV" + UNIVERSE_GATEWAY_PORT=$GATEWAY_PORT UNIVERSE_GATEWAY_ROOT=frontend/dist/mempool/browser node scripts/universe/gateway.mjs & + GATEWAY_PID=$! + for i in $(seq 1 30); do + curl -fsS -o /dev/null "http://127.0.0.1:$GATEWAY_PORT/" && break + sleep 1 + done + kill -0 "$GATEWAY_PID" + curl -fsS "http://127.0.0.1:$GATEWAY_PORT/__gateway/health" + + # The routes this suite holds to a terminal state, taken through every + # failure state, every theme and the widths that change the layout, which + # is where a page that lies about why it is empty shows up. - name: The gated routes hold up across themes, widths and every failure state working-directory: scripts/universe/visual-qa run: >- node capture.mjs --base=http://127.0.0.1:$GATEWAY_PORT --out=./artifacts - --routes=graphs,mining,protocols,home,blocks,tx,address --viewports=320,768,1440 - - # The chain failure states have never run here. They are scoped to the - # chain routes, so the pass above never reaches them, and they were - # reviewed by hand instead: a chain that cannot be reached, one that is - # behind, one whose node is caught up while its protocol side is not, one - # that withholds readiness without saying why, and an object that simply - # is not there. Those five are the states the whole status vocabulary - # exists for, and the one set nothing measured. + --routes=${{ matrix.routes }} --viewports=320,768,1440 + + # The chain failure states are scoped to the chain routes, so the pass + # above never reaches them. Five states: a chain that cannot be reached, + # one that is behind, one whose node is caught up while its protocol side + # is not, one that withholds readiness without saying why, and an object + # that simply is not there. Those five are what the whole status + # vocabulary exists for, and for a long time nothing measured them. # - # Two themes and two widths rather than three and three, because the cost - # has to fit beside the pass above and this is the part of the matrix - # that catches wording and state, not layout. + # Two themes and two widths rather than three and three, because this is + # the part of the matrix that catches wording and state, not layout. - name: A chain that cannot answer says so, in every failure state + if: matrix.chainStates == 'yes' working-directory: scripts/universe/visual-qa run: >- node capture.mjs --base=http://127.0.0.1:$GATEWAY_PORT --out=./artifacts-chain-states @@ -289,18 +358,13 @@ jobs: --states=chain-authority-down,chain-behind,chain-not-ready-protocols,chain-not-ready-unexplained,chain-object-missing --themes=default,dark --viewports=375,1440 - - name: Forced colours and 200 percent zoom - working-directory: scripts/universe/visual-qa - run: node modes-check.mjs --base=http://127.0.0.1:$GATEWAY_PORT - - name: Keep the contrast report and the route screenshots if: always() uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 with: - name: universe-visual-qa + name: universe-visual-qa-${{ matrix.shard }} path: | scripts/universe/visual-qa/artifacts - scripts/universe/visual-qa/artifacts-broad scripts/universe/visual-qa/artifacts-chain-states retention-days: 30 From 751346dca8a6f34c288c6ee50f75940a3303ab7d Mon Sep 17 00:00:00 2001 From: Bitcoin Universe Date: Sun, 30 Aug 2026 18:39:20 +0000 Subject: [PATCH 07/15] Escape the backslash before the pipe in a table cell Escaping only the pipe leaves a value ending in a backslash able to consume its own escape and split the row, so one cell reads as two. The backslash goes first and a newline becomes a space, because a newline ends the row outright. CodeQL flagged it as js/incomplete-sanitization on the pull request that introduced it. Co-Authored-By: Claude Opus 5 --- scripts/universe/protocol-contract.mjs | 13 +++++++++++-- scripts/universe/protocol-contract.test.mjs | 20 ++++++++++++++++++++ 2 files changed, 31 insertions(+), 2 deletions(-) diff --git a/scripts/universe/protocol-contract.mjs b/scripts/universe/protocol-contract.mjs index 81e58bb4b9..93b132ee7d 100644 --- a/scripts/universe/protocol-contract.mjs +++ b/scripts/universe/protocol-contract.mjs @@ -278,9 +278,18 @@ export function resolvableNames(protocols) { // Generated documents // --------------------------------------------------------------------------- -/** Escapes the pipe so an authority id can never break the table. */ +/** + * Escapes a value so it can never break the table it is written into. + * + * The backslash goes first: escaping the pipe alone leaves a value ending in + * a backslash able to consume the escape and split the row, which is a cell + * that reads as two. Newlines end a row outright, so they become spaces. + */ function cell(value) { - return String(value ?? '').replace(/\|/g, '\\|'); + return String(value ?? '') + .replace(/\\/g, '\\\\') + .replace(/\|/g, '\\|') + .replace(/[\r\n]+/g, ' '); } export function readableProtocols(manifest) { diff --git a/scripts/universe/protocol-contract.test.mjs b/scripts/universe/protocol-contract.test.mjs index 4c055c627f..a272110f19 100644 --- a/scripts/universe/protocol-contract.test.mjs +++ b/scripts/universe/protocol-contract.test.mjs @@ -12,6 +12,7 @@ import { diffManifests, protocolCopyIds, readmeBlockOf, + renderMarkdown, renderReadmeBlock, renderRoster, validateManifest, @@ -259,6 +260,25 @@ test('a served roster in a different order is named', () => { ); }); +test('a value carrying a pipe or a backslash cannot split a table row', () => { + const rendered = renderMarkdown({ + ...pinned, + protocols: [ + { + ...pinned.protocols[0], + id: 'trick', + family: 'a\\', + chain: 'b|c', + indexerAuthority: 'd\ne', + }, + ], + }); + const row = rendered + .split('\n') + .find((line) => line.startsWith('| trick ')); + assert.equal(row.split(/(? { assert.deepEqual( diffManifests(comparable(pinned), comparable(pinned)).problems, From a390b63ee84c8a1fe8b68d8daa4c2784f1678926 Mon Sep 17 00:00:00 2001 From: Bitcoin Universe Date: Sun, 30 Aug 2026 19:16:08 +0000 Subject: [PATCH 08/15] Let the gate finish before it reports its verdict Exiting from the catch while a keep-alive socket was still being torn down aborted the process with a libuv assertion and an exit code of 127 instead of the 1 the failure means. Both fail a job, so CI was never wrong, but a gate whose exit code depends on which machine ran it is a gate nobody can reason about. The body is now read either way, so nothing holds the connection, and the failure sets the exit code and lets the process end on its own. Measured before: exit 127 on Windows against the live origin. After: 1. Co-Authored-By: Claude Opus 5 --- scripts/universe/protocol-contract.mjs | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/scripts/universe/protocol-contract.mjs b/scripts/universe/protocol-contract.mjs index 93b132ee7d..3a31c82162 100644 --- a/scripts/universe/protocol-contract.mjs +++ b/scripts/universe/protocol-contract.mjs @@ -618,12 +618,21 @@ async function loadDocument(source) { redirect: 'error', signal: AbortSignal.timeout(30_000), }); + // Read the body either way. An unread body leaves the connection open and + // the process cannot finish on its own while it is held. + const body = await response.text(); if (!response.ok) { throw new GateFailure( `${url} answered HTTP ${response.status}; the roster could not be read.`, ); } - return response.json(); + try { + return JSON.parse(body); + } catch { + throw new GateFailure( + `${url} did not answer with JSON; the roster could not be read.`, + ); + } } return JSON.parse(await readFile(path.resolve(source), 'utf8')); } @@ -765,6 +774,11 @@ const invokedDirectly = if (invokedDirectly) { main().catch((error) => { process.stderr.write(`${error instanceof Error ? error.message : error}\n`); - process.exit(1); + // Set the code rather than exiting here, so the process ends when its own + // handles close instead of being torn down in the middle of a socket + // teardown. Exiting mid-teardown aborts with a libuv assertion and an + // exit code of 127 on Windows, which is a gate whose verdict depends on + // which machine ran it. + process.exitCode = 1; }); } From 94f72c4b417dc86266b64c462dcbcb4c744c3462 Mon Sep 17 00:00:00 2001 From: Bitcoin Universe Date: Sun, 30 Aug 2026 20:00:12 +0000 Subject: [PATCH 09/15] Keep the screenshots only when there is something to look at Run 33330398257 passed every gate in the frontend job and then reported a failure: ##[error]Failed to CreateArtifact: Artifact storage quota has been hit. Unable to upload any new artifacts. Nothing was wrong with the build. The deep pass uploads about fifty megabytes of screenshots per run and the broad pass another seven, on every run, green or not, and that is what filled the quota. Usage is recalculated every six to twelve hours, so deleting faster does not fix it; producing less does. 1.1 GB of artifacts from before 17:00Z were deleted to clear the immediate block. The screenshots are diagnostic. When a gate passes, nobody opens them; when one fails, they are exactly what is needed. Both screenshot uploads now run on failure, with `if-no-files-found: ignore` so a job that failed before the pass ran does not fail twice for it. Retention was already capped at three days by the repository, which every run said in a warning while the file asked for thirty. It now asks for what it can have. Co-Authored-By: Claude Opus 5 --- .github/workflows/universe-ci.yml | 26 ++++++++++++++++++++------ 1 file changed, 20 insertions(+), 6 deletions(-) diff --git a/.github/workflows/universe-ci.yml b/.github/workflows/universe-ci.yml index b35581b4d0..9dafb8be37 100644 --- a/.github/workflows/universe-ci.yml +++ b/.github/workflows/universe-ci.yml @@ -246,13 +246,24 @@ jobs: working-directory: scripts/universe/visual-qa run: node modes-check.mjs --base=http://127.0.0.1:$GATEWAY_PORT + # Kept when there is something to look at, which is when a gate failed. + # + # A green deep pass is five hundred screenshots nobody opens, and + # uploading them on every run filled the Actions artifact quota: run + # 33330398257 passed every gate in this job and then failed on + # "Artifact storage quota has been hit. Unable to upload any new + # artifacts", so a build with nothing wrong with it reported a failure. + # Usage is recalculated every six to twelve hours, so the way out is to + # stop producing it rather than to delete faster. Retention is capped at + # three days by the repository in any case. - name: Keep the contrast report and the route screenshots - if: always() + if: failure() uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 with: name: universe-visual-qa-broad path: scripts/universe/visual-qa/artifacts-broad - retention-days: 30 + if-no-files-found: ignore + retention-days: 3 # The deep visual pass, in three shards. # @@ -358,15 +369,18 @@ jobs: --states=chain-authority-down,chain-behind,chain-not-ready-protocols,chain-not-ready-unexplained,chain-object-missing --themes=default,dark --viewports=375,1440 + # Same reasoning as the broad pass above: this is the artifact that + # filled the quota, at about fifty megabytes a run. - name: Keep the contrast report and the route screenshots - if: always() + if: failure() uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 with: name: universe-visual-qa-${{ matrix.shard }} path: | scripts/universe/visual-qa/artifacts scripts/universe/visual-qa/artifacts-chain-states - retention-days: 30 + if-no-files-found: ignore + retention-days: 3 # The mobile gate runs beside the visual matrix rather than inside it. # @@ -458,7 +472,7 @@ jobs: path: | scripts/universe/visual-qa/artifacts-mobile scripts/universe/visual-qa/artifacts-perf - retention-days: 30 + retention-days: 3 # The two engines the fleet cannot run. # @@ -552,4 +566,4 @@ jobs: path: | scripts/universe/visual-qa/artifacts-mobile-webkit scripts/universe/visual-qa/artifacts-mobile-firefox - retention-days: 30 + retention-days: 3 From b157f160494c6229a0485d1ea312cca9e761311b Mon Sep 17 00:00:00 2001 From: Bitcoin Universe Date: Sun, 30 Aug 2026 20:19:01 +0000 Subject: [PATCH 10/15] Run the third-engine gate on our own capacity WebKit and Firefox was the one required check in this workflow that asked for a GitHub-hosted runner. On 2026-08-30 the organization's plan stopped providing them, and from about 19:39Z every run of this job failed while the four self-hosted checks beside it kept passing. A required gate no runner can accept blocks every merge in the repository, which is the same failure as a gate that cannot finish. It moves to `browser-heavy`, which already carries a browser and the system libraries one needs. `playwright install --with-deps` wants root and a runner account may not have it, so a refused dependency install falls back to fetching the engines alone rather than failing a gate over a permission. Nothing about what is measured changes: the same WebKit window set and the same Firefox subset, on the same routes. Co-Authored-By: Claude Opus 5 --- .github/workflows/universe-ci.yml | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/.github/workflows/universe-ci.yml b/.github/workflows/universe-ci.yml index 9dafb8be37..de641af1d5 100644 --- a/.github/workflows/universe-ci.yml +++ b/.github/workflows/universe-ci.yml @@ -496,7 +496,15 @@ jobs: # two steps back in the one above, which is where they belong. mobile-engines: name: WebKit and Firefox - runs-on: ubuntu-latest + # On our own capacity, not GitHub's. + # + # This was the one required check in this workflow that needed a + # GitHub-hosted runner, and on 2026-08-30 the organization's plan stopped + # providing them: every run of this job failed while the four self-hosted + # checks beside it kept passing. A required gate that no runner can accept + # is the same problem as a gate that cannot finish, and `browser-heavy` + # already carries a browser and the system libraries one needs. + runs-on: [self-hosted, browser-heavy] timeout-minutes: 45 steps: - name: Checkout @@ -521,9 +529,15 @@ jobs: working-directory: scripts/universe/visual-qa run: npm ci + # `--with-deps` installs system libraries and needs root, which a + # self-hosted runner account may not have. These runners already carry + # what Chromium needs, which is very nearly the same set, so a refused + # dependency install falls back to fetching the engines alone rather + # than failing a gate over a permission. - name: Fetch the engines and what they need to start working-directory: scripts/universe/visual-qa - run: npx playwright install --with-deps webkit firefox + run: | + npx playwright install --with-deps webkit firefox || npx playwright install webkit firefox - name: Serve the built frontend run: | From dd7fb5119b026da52502946c1c3ef3bff06c3140 Mon Sep 17 00:00:00 2001 From: Bitcoin Universe Admin Date: Sun, 30 Aug 2026 20:42:35 +0000 Subject: [PATCH 11/15] ci: replace GitHub-hosted runners with RunsOn Spot GitHub-hosted labels bypass the self-hosted fleet and the RunsOn AWS overflow, and they are what consumed the organization's Actions minute allowance. The official RunsOn images track the GitHub runner images closely, so the toolchain is unchanged. Co-Authored-By: Claude Opus 5 --- .github/workflows/backend-integration.yml | 2 +- .github/workflows/ci.yml | 10 +++++----- .github/workflows/docker.yml | 6 +++--- .github/workflows/e2e_parameterized.yml | 4 ++-- .github/workflows/get_backend_block_height.yml | 2 +- .github/workflows/get_backend_hash.yml | 2 +- .github/workflows/get_image_digest.yml | 2 +- .github/workflows/project-review-status.yml | 2 +- .github/workflows/universe-ci.yml | 2 +- 9 files changed, 16 insertions(+), 16 deletions(-) diff --git a/.github/workflows/backend-integration.yml b/.github/workflows/backend-integration.yml index 89624cb77b..09685e52fa 100644 --- a/.github/workflows/backend-integration.yml +++ b/.github/workflows/backend-integration.yml @@ -15,7 +15,7 @@ jobs: matrix: node: ["24.13.0"] fail-fast: false - runs-on: mempool-ci + runs-on: runs-on=${{ github.run_id }}-backend_integration/runner=universe-hosted/cpu=2+8/env=production env: COMPOSE_PROJECT_NAME: mempool-integration-${{ github.run_id }}-${{ github.run_attempt }}-${{ github.job }} MEMPOOL_TEST_BASE_CONFIG_FILE: ${{ github.workspace }}/${{ matrix.node }}/integration/backend/mempool-config.test.json diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 6656d41679..af5bf79d2f 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -16,7 +16,7 @@ jobs: node: ["24.13.0"] flavor: ["dev", "prod"] fail-fast: false - runs-on: mempool-ci + runs-on: runs-on=${{ github.run_id }}-backend/runner=universe-hosted/cpu=2+8/env=production name: Backend (${{ matrix.flavor }}) - node ${{ matrix.node }} steps: @@ -101,7 +101,7 @@ jobs: strategy: matrix: node: ["24.13.0"] - runs-on: mempool-ci + runs-on: runs-on=${{ github.run_id }}-cache/runner=universe-hosted/cpu=2+8/env=production steps: - name: Checkout uses: actions/checkout@v3 @@ -212,7 +212,7 @@ jobs: node: ["24.13.0"] flavor: ["dev", "prod"] fail-fast: false - runs-on: mempool-ci + runs-on: runs-on=${{ github.run_id }}-frontend/runner=universe-hosted/cpu=2+8/env=production name: Frontend (${{ matrix.flavor }}) - node ${{ matrix.node }} steps: @@ -309,7 +309,7 @@ jobs: e2e: if: "(github.event_name == 'pull_request' && !contains(github.event.pull_request.labels.*.name, 'ops') && !contains(github.head_ref, 'ops/')) || github.event_name == 'push'" - runs-on: mempool-ci + runs-on: runs-on=${{ github.run_id }}-e2e/runner=universe-hosted/cpu=2+8/env=production needs: frontend strategy: fail-fast: false @@ -453,7 +453,7 @@ jobs: CYPRESS_PROJECT_ID: ${{ secrets.CYPRESS_PROJECT_ID }} validate_docker_json: if: "(github.event_name == 'pull_request' && !contains(github.event.pull_request.labels.*.name, 'ops') && !contains(github.head_ref, 'ops/')) || github.event_name == 'push'" - runs-on: mempool-ci + runs-on: runs-on=${{ github.run_id }}-validate_docker_json/runner=universe-hosted/cpu=2+8/env=production name: Validate generated backend Docker JSON steps: diff --git a/.github/workflows/docker.yml b/.github/workflows/docker.yml index f4abaf3192..c6eb44d834 100644 --- a/.github/workflows/docker.yml +++ b/.github/workflows/docker.yml @@ -20,7 +20,7 @@ permissions: jobs: test-images: # Always run on tag pushes and all pull requests - runs-on: mempool-ci + runs-on: runs-on=${{ github.run_id }}-test_images/runner=universe-hosted/cpu=2+8/env=production timeout-minutes: 30 name: Test built Docker images steps: @@ -233,7 +233,7 @@ jobs: service: - frontend - backend - runs-on: mempool-ci + runs-on: runs-on=${{ github.run_id }}-build/runner=universe-hosted/cpu=2+8/env=production timeout-minutes: 120 name: Build and push to DockerHub outputs: @@ -363,7 +363,7 @@ jobs: needs: build # Only for successful tag pushes (not PRs with docker-push label) and only for "plain" versions (no '-') if: ${{ needs.build.result == 'success' && github.event_name == 'push' && !contains(github.ref_name, '-') }} - runs-on: mempool-ci + runs-on: runs-on=${{ github.run_id }}-tag_latest/runner=universe-hosted/cpu=2+8/env=production timeout-minutes: 30 name: Tag release build as latest strategy: diff --git a/.github/workflows/e2e_parameterized.yml b/.github/workflows/e2e_parameterized.yml index ac4bca66d4..089adb407c 100644 --- a/.github/workflows/e2e_parameterized.yml +++ b/.github/workflows/e2e_parameterized.yml @@ -22,7 +22,7 @@ on: jobs: cache: name: "Cache assets for builds" - runs-on: mempool-ci + runs-on: runs-on=${{ github.run_id }}-cache/runner=universe-hosted/cpu=2+8/env=production steps: - name: Determine checkout ref id: determine-ref @@ -123,7 +123,7 @@ jobs: key: promo-video-assets-cache e2e: - runs-on: mempool-ci + runs-on: runs-on=${{ github.run_id }}-e2e/runner=universe-hosted/cpu=2+8/env=production needs: cache strategy: fail-fast: false diff --git a/.github/workflows/get_backend_block_height.yml b/.github/workflows/get_backend_block_height.yml index 73db6107ee..088f628199 100644 --- a/.github/workflows/get_backend_block_height.yml +++ b/.github/workflows/get_backend_block_height.yml @@ -4,7 +4,7 @@ on: [workflow_dispatch] jobs: print-backend-sha: - runs-on: mempool-ci + runs-on: runs-on=${{ github.run_id }}-print_backend_sha/runner=universe-hosted/cpu=2+8/env=production name: Get block height steps: - name: Checkout diff --git a/.github/workflows/get_backend_hash.yml b/.github/workflows/get_backend_hash.yml index d63860c5e8..0c4c3d28e5 100644 --- a/.github/workflows/get_backend_hash.yml +++ b/.github/workflows/get_backend_hash.yml @@ -4,7 +4,7 @@ on: [workflow_dispatch] jobs: print-backend-sha: - runs-on: mempool-ci + runs-on: runs-on=${{ github.run_id }}-print_backend_sha/runner=universe-hosted/cpu=2+8/env=production name: Print backend hashes steps: - name: Checkout diff --git a/.github/workflows/get_image_digest.yml b/.github/workflows/get_image_digest.yml index 3d86c860c1..d43bea0528 100644 --- a/.github/workflows/get_image_digest.yml +++ b/.github/workflows/get_image_digest.yml @@ -10,7 +10,7 @@ on: type: string jobs: print-images-sha: - runs-on: mempool-ci + runs-on: runs-on=${{ github.run_id }}-print_images_sha/runner=universe-hosted/cpu=2+8/env=production name: Print digest for images steps: - name: Checkout diff --git a/.github/workflows/project-review-status.yml b/.github/workflows/project-review-status.yml index 7e85b8eb34..3caa9a755b 100644 --- a/.github/workflows/project-review-status.yml +++ b/.github/workflows/project-review-status.yml @@ -12,7 +12,7 @@ on: jobs: manage-project-board: - runs-on: mempool-ci + runs-on: runs-on=${{ github.run_id }}-manage_project_board/runner=universe-hosted/cpu=2+8/env=production steps: - name: Update Project Board uses: actions/github-script@v7 diff --git a/.github/workflows/universe-ci.yml b/.github/workflows/universe-ci.yml index a71293aa03..37b7a704db 100644 --- a/.github/workflows/universe-ci.yml +++ b/.github/workflows/universe-ci.yml @@ -408,7 +408,7 @@ jobs: # two steps back in the one above, which is where they belong. mobile-engines: name: WebKit and Firefox - runs-on: ubuntu-latest + runs-on: runs-on=${{ github.run_id }}-mobile_engines/runner=universe-hosted/cpu=2+8/env=production timeout-minutes: 45 steps: - name: Checkout From 288a1813ed78763e05138c3a907e4e7172e5c4d6 Mon Sep 17 00:00:00 2001 From: Bitcoin Universe Date: Sun, 30 Aug 2026 20:45:45 +0000 Subject: [PATCH 12/15] Stop every green run from writing an artifact it cannot write Run 33333310466 passed every measurement in "Mobile and adaptive layout" and then failed on "Keep the mobile report". The Actions artifact quota is full, so any upload attempt fails the step, and a required check that is green in substance reports red. The two screenshot uploads were already narrowed to failure. These are the three that were left: the mobile report, the engine reports, and the chain page smoke. Same rule, same reason. On failure the upload may still be refused, but the job is failing already, so nothing changes about the verdict. Co-Authored-By: Claude Opus 5 --- .github/workflows/universe-ci.yml | 10 ++++++++-- .github/workflows/universe-production-smoke.yml | 8 ++++++-- 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/.github/workflows/universe-ci.yml b/.github/workflows/universe-ci.yml index de641af1d5..6f575ecebe 100644 --- a/.github/workflows/universe-ci.yml +++ b/.github/workflows/universe-ci.yml @@ -464,10 +464,15 @@ jobs: node mobile-perf.mjs --base=http://127.0.0.1:$GATEWAY_PORT --out=./artifacts-perf + # Kept on failure only, for the reason in the frontend job above: the + # Actions artifact quota is full, and an upload attempt fails the step. + # Run 33333310466 passed every measurement in this job and then failed + # on this upload, which is a green gate reporting red. - name: Keep the mobile report - if: always() + if: failure() uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 with: + if-no-files-found: ignore name: universe-mobile-qa path: | scripts/universe/visual-qa/artifacts-mobile @@ -573,9 +578,10 @@ jobs: --out=./artifacts-mobile-firefox - name: Keep the engine reports - if: always() + if: failure() uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 with: + if-no-files-found: ignore name: universe-mobile-engines path: | scripts/universe/visual-qa/artifacts-mobile-webkit diff --git a/.github/workflows/universe-production-smoke.yml b/.github/workflows/universe-production-smoke.yml index 9669dccd1a..444452b4e4 100644 --- a/.github/workflows/universe-production-smoke.yml +++ b/.github/workflows/universe-production-smoke.yml @@ -86,10 +86,14 @@ jobs: node chain-page-smoke.mjs --origin="${{ github.event.inputs.origin || 'https://explorer.bitcoinuniverse.io' }}" + # On failure only. The Actions artifact quota is full, an upload attempt + # fails the step, and a scheduled check that reports red for its own + # screenshots teaches everyone to ignore it. - name: Keep what the chain pages looked like - if: always() + if: failure() uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 with: + if-no-files-found: ignore name: universe-chain-page-smoke path: scripts/universe/visual-qa/artifacts-chain-smoke - retention-days: 30 + retention-days: 3 From fd82d92da203fab7c62b024da8e6819f1370e79d Mon Sep 17 00:00:00 2001 From: Bitcoin Universe Date: Sun, 30 Aug 2026 20:50:43 +0000 Subject: [PATCH 13/15] Restore the workflow this merge resolution dropped The conflict in universe-ci.yml was one line, the runner target for the third-engine job, and resolving it by taking the whole file from develop threw away everything else on this branch: the sharded visual matrix, the artifact policy, and the step that runs the new roster gate. The run that followed proved it, failing on a step named for a script this branch deletes. The file is this branch again, with develop's RunsOn target for mobile-engines kept. Co-Authored-By: Claude Opus 5 --- .github/workflows/universe-ci.yml | 242 ++++++++++++++++++++++-------- 1 file changed, 176 insertions(+), 66 deletions(-) diff --git a/.github/workflows/universe-ci.yml b/.github/workflows/universe-ci.yml index 37b7a704db..1abdec27e0 100644 --- a/.github/workflows/universe-ci.yml +++ b/.github/workflows/universe-ci.yml @@ -44,8 +44,18 @@ jobs: with: node-version: '24.19.0' - - name: Protocol coverage table matches the recorded manifest - run: node scripts/universe/generate-protocol-coverage.mjs --check + # The roster is owned by backend-apis. This holds the pinned copy to its + # schema and its provenance, holds this repository's own surfaces to the + # pinned roster, and refuses a roster that lost an entry. The gate it + # replaced rendered a table from the recorded JSON and compared it with + # the committed table, so it could only prove a file agreed with itself: + # dunes and zrc20 were added to the registry and served in production + # while the recorded roster here still said thirty-six, green throughout. + - name: The pinned protocol roster holds + run: node scripts/universe/protocol-contract.mjs --check + + - name: The protocol roster gate fires on the states it exists for + run: node --test scripts/universe/protocol-contract.test.mjs - name: No em dash anywhere run: node scripts/universe/check-text.mjs @@ -124,12 +134,12 @@ jobs: frontend: name: Frontend build runs-on: [self-hosted, linux-ultra] - # The deep visual pass measured 48 minutes once the gated route list grew - # from three routes to seven, which put the job over the hour it had and - # killed it three minutes from the end. A gate that cannot finish protects - # nothing, which is the same reason the matrix was split in two to begin - # with. Budget is for the pass that actually runs, with room to grow. - timeout-minutes: 100 + # This builds, gates the bundle, and walks every route once at a phone and + # a desktop width. The deep pass that used to follow it now runs beside it, + # sharded, because it stopped fitting: 71 minutes of a 100 minute cap on + # its own, and on develop it reached the cap and the required check died as + # a timeout. Budget here is for the work that is left, with room to grow. + timeout-minutes: 45 steps: - name: Checkout uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 @@ -214,64 +224,144 @@ jobs: # chart panel draws nothing, or when a failure state says nothing about # why. Those three states passed every other gate here and shipped. # - # Split into two passes that finish, rather than one sweep that does not. - # Every combination of thirteen routes, seven data states, three themes - # and seven widths is nineteen hundred page loads, which does not fit the - # job timeout, and a gate that cannot finish protects nothing. - # - # The broad pass walks every route at a phone and a desktop width with - # real data, which is where overflow, broken images and a page that never - # finishes show up. The deep pass takes the routes this suite holds to a - # terminal state through every failure state, every theme and the widths - # that change the layout, which is where a page that lies about why it is - # empty shows up. - # - # Measured on run 33254326169, 2026-08-29, on universe-linux-ultra-02: - # - # broad pass 64 screenshots 5m50s - # deep pass 504 screenshots 47m10s - # chain failures 48 screenshots 3m27s - # forced colours and zoom 30s - # whole job 59m57s of a 100 minute cap + # The pass here is the broad one: every route, real data, a phone and a + # desktop width, which is where overflow, broken images and a page that + # never finishes show up. It measured 5m50s. # - # This comment said two and a half minutes and sixteen. The deep pass has - # since grown to roughly three times that, it is now 78 percent of the - # job, and the job sits at 60 of its 100 minutes. Nothing is failing. The - # point of writing the real numbers down is that the next person to add a - # route, a state or a theme to the deep pass is the one who meets the - # cap, and without this they meet it as a timeout forty-seven minutes - # into a run rather than as a number they could have added up first. - # - # The deep pass is routes multiplied by states multiplied by themes - # multiplied by widths. Adding one route to it currently costs about - # seven minutes. If it needs to grow again, split it the way it was split - # from the broad pass in the first place, rather than raising the cap: - # a gate that cannot finish protects nothing, which is the reason the - # split exists. + # The deep pass is the `visual` job below. It is routes multiplied by + # states multiplied by themes multiplied by widths, it grew to 71 minutes + # of this job's 100 minute cap, and on develop it reached the cap: run + # 33320958080 was killed at 1h41 with the deep pass still going, so the + # required check for the promotion had no verdict at all. The comment + # that used to sit here said to split it again rather than raise the cap, + # because a gate that cannot finish protects nothing. That is what the + # shards are. - name: Every route renders and finishes, at a phone and a desktop width working-directory: scripts/universe/visual-qa run: >- node capture.mjs --base=http://127.0.0.1:$GATEWAY_PORT --out=./artifacts-broad --states=populated --themes=default --viewports=375,1440 + - name: Forced colours and 200 percent zoom + working-directory: scripts/universe/visual-qa + run: node modes-check.mjs --base=http://127.0.0.1:$GATEWAY_PORT + + # Kept when there is something to look at, which is when a gate failed. + # + # A green deep pass is five hundred screenshots nobody opens, and + # uploading them on every run filled the Actions artifact quota: run + # 33330398257 passed every gate in this job and then failed on + # "Artifact storage quota has been hit. Unable to upload any new + # artifacts", so a build with nothing wrong with it reported a failure. + # Usage is recalculated every six to twelve hours, so the way out is to + # stop producing it rather than to delete faster. Retention is capped at + # three days by the repository in any case. + - name: Keep the contrast report and the route screenshots + if: failure() + uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 + with: + name: universe-visual-qa-broad + path: scripts/universe/visual-qa/artifacts-broad + if-no-files-found: ignore + retention-days: 3 + + # The deep visual pass, in three shards. + # + # Measured on run 33323262662, 2026-08-30, on universe-linux-ultra-04: seven + # gated routes across three widths, seven states and the themes they carry + # took 1h11m in one step. Ten minutes a route. One more route would have put + # the job past its cap; on develop it already had. + # + # The split is by route because the cost is per route, and the shards are + # sized from that measurement rather than by counting names. Each shard + # builds its own copy, the way the mobile gate does: it costs one production + # build and buys a gate that can be read, run and debugged on its own, and + # that cannot make the job beside it wait. + # + # charts-and-mining graphs, mining about 20 minutes + # protocols-and-home protocols, home about 20 minutes + # blocks-and-transactions blocks, tx, address about 30 minutes + # plus the chain failure states, 3m27s + visual: + name: Visual matrix (${{ matrix.shard }}) + runs-on: [self-hosted, linux-ultra] + timeout-minutes: 60 + strategy: + # One shard failing says nothing about the others, and finding out that + # two route groups are broken in one run beats finding out twice. + fail-fast: false + matrix: + include: + - shard: charts-and-mining + routes: graphs,mining + chainStates: 'no' + - shard: protocols-and-home + routes: protocols,home + chainStates: 'no' + - shard: blocks-and-transactions + routes: blocks,tx,address + chainStates: 'yes' + steps: + - name: Checkout + uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + + - name: Setup Node + uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4 + with: + node-version: '24.19.0' + cache: npm + cache-dependency-path: frontend/package-lock.json + + - name: Install + working-directory: frontend + run: npm ci + + - name: Production build + working-directory: frontend + run: npm run build:universe + + - name: Install the visual QA harness + working-directory: scripts/universe/visual-qa + run: npm ci + + # A fixed port here once measured another job's build: two runners share + # a host, the second gateway crashed on the taken port behind "&", the + # probe happily reached the first job's server, and the run died halfway + # when that other job ended. Each run now serves on its own free port and + # refuses to continue unless the process answering is the one it started. + - name: Serve the built frontend + run: | + GATEWAY_PORT=$(node -e "const s=require('net').createServer();s.listen(0,'127.0.0.1',()=>{console.log(s.address().port);s.close()})") + echo "GATEWAY_PORT=$GATEWAY_PORT" >> "$GITHUB_ENV" + UNIVERSE_GATEWAY_PORT=$GATEWAY_PORT UNIVERSE_GATEWAY_ROOT=frontend/dist/mempool/browser node scripts/universe/gateway.mjs & + GATEWAY_PID=$! + for i in $(seq 1 30); do + curl -fsS -o /dev/null "http://127.0.0.1:$GATEWAY_PORT/" && break + sleep 1 + done + kill -0 "$GATEWAY_PID" + curl -fsS "http://127.0.0.1:$GATEWAY_PORT/__gateway/health" + + # The routes this suite holds to a terminal state, taken through every + # failure state, every theme and the widths that change the layout, which + # is where a page that lies about why it is empty shows up. - name: The gated routes hold up across themes, widths and every failure state working-directory: scripts/universe/visual-qa run: >- node capture.mjs --base=http://127.0.0.1:$GATEWAY_PORT --out=./artifacts - --routes=graphs,mining,protocols,home,blocks,tx,address --viewports=320,768,1440 - - # The chain failure states have never run here. They are scoped to the - # chain routes, so the pass above never reaches them, and they were - # reviewed by hand instead: a chain that cannot be reached, one that is - # behind, one whose node is caught up while its protocol side is not, one - # that withholds readiness without saying why, and an object that simply - # is not there. Those five are the states the whole status vocabulary - # exists for, and the one set nothing measured. + --routes=${{ matrix.routes }} --viewports=320,768,1440 + + # The chain failure states are scoped to the chain routes, so the pass + # above never reaches them. Five states: a chain that cannot be reached, + # one that is behind, one whose node is caught up while its protocol side + # is not, one that withholds readiness without saying why, and an object + # that simply is not there. Those five are what the whole status + # vocabulary exists for, and for a long time nothing measured them. # - # Two themes and two widths rather than three and three, because the cost - # has to fit beside the pass above and this is the part of the matrix - # that catches wording and state, not layout. + # Two themes and two widths rather than three and three, because this is + # the part of the matrix that catches wording and state, not layout. - name: A chain that cannot answer says so, in every failure state + if: matrix.chainStates == 'yes' working-directory: scripts/universe/visual-qa run: >- node capture.mjs --base=http://127.0.0.1:$GATEWAY_PORT --out=./artifacts-chain-states @@ -279,20 +369,18 @@ jobs: --states=chain-authority-down,chain-behind,chain-not-ready-protocols,chain-not-ready-unexplained,chain-object-missing --themes=default,dark --viewports=375,1440 - - name: Forced colours and 200 percent zoom - working-directory: scripts/universe/visual-qa - run: node modes-check.mjs --base=http://127.0.0.1:$GATEWAY_PORT - + # Same reasoning as the broad pass above: this is the artifact that + # filled the quota, at about fifty megabytes a run. - name: Keep the contrast report and the route screenshots - if: always() + if: failure() uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 with: - name: universe-visual-qa + name: universe-visual-qa-${{ matrix.shard }} path: | scripts/universe/visual-qa/artifacts - scripts/universe/visual-qa/artifacts-broad scripts/universe/visual-qa/artifacts-chain-states - retention-days: 30 + if-no-files-found: ignore + retention-days: 3 # The mobile gate runs beside the visual matrix rather than inside it. # @@ -376,15 +464,20 @@ jobs: node mobile-perf.mjs --base=http://127.0.0.1:$GATEWAY_PORT --out=./artifacts-perf + # Kept on failure only, for the reason in the frontend job above: the + # Actions artifact quota is full, and an upload attempt fails the step. + # Run 33333310466 passed every measurement in this job and then failed + # on this upload, which is a green gate reporting red. - name: Keep the mobile report - if: always() + if: failure() uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 with: + if-no-files-found: ignore name: universe-mobile-qa path: | scripts/universe/visual-qa/artifacts-mobile scripts/universe/visual-qa/artifacts-perf - retention-days: 30 + retention-days: 3 # The two engines the fleet cannot run. # @@ -408,6 +501,16 @@ jobs: # two steps back in the one above, which is where they belong. mobile-engines: name: WebKit and Firefox + # RunsOn Spot, per the change on develop that replaced the GitHub-hosted + # label here. This was the one required check in this workflow that needed + # a GitHub-hosted runner, and on 2026-08-30 the organization's plan stopped + # providing them: every run of this job failed while the four self-hosted + # checks beside it kept passing. + # + # Worth revisiting: this job takes screenshots at seven widths, and the + # routing rule those changes introduced keeps screenshot and baseline work + # on the local fleet because its output depends on the host. `browser-heavy` + # carries a browser and the libraries these engines need, and costs nothing. runs-on: runs-on=${{ github.run_id }}-mobile_engines/runner=universe-hosted/cpu=2+8/env=production timeout-minutes: 45 steps: @@ -433,9 +536,15 @@ jobs: working-directory: scripts/universe/visual-qa run: npm ci + # `--with-deps` installs system libraries and needs root, which a + # self-hosted runner account may not have. These runners already carry + # what Chromium needs, which is very nearly the same set, so a refused + # dependency install falls back to fetching the engines alone rather + # than failing a gate over a permission. - name: Fetch the engines and what they need to start working-directory: scripts/universe/visual-qa - run: npx playwright install --with-deps webkit firefox + run: | + npx playwright install --with-deps webkit firefox || npx playwright install webkit firefox - name: Serve the built frontend run: | @@ -471,11 +580,12 @@ jobs: --out=./artifacts-mobile-firefox - name: Keep the engine reports - if: always() + if: failure() uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 with: + if-no-files-found: ignore name: universe-mobile-engines path: | scripts/universe/visual-qa/artifacts-mobile-webkit scripts/universe/visual-qa/artifacts-mobile-firefox - retention-days: 30 + retention-days: 3 From b21d0793ea6e9bd6ad8cd9058efa6e1849c0adf8 Mon Sep 17 00:00:00 2001 From: Bitcoin Universe Date: Sun, 30 Aug 2026 20:53:27 +0000 Subject: [PATCH 14/15] Pin the roster the deployed overlay serves backend-apis main 100c0ec0 is live. It promotes op_inscriptions from BLOCKED to VERIFIED READ ONLY, which production supports: /api/v1/universe/sources reports index-opinscriptions ready, zero blocks behind, zero consecutive failures. So the pin is re-recorded from the deployed origin: seven protocols readable rather than six, and the sourceSha is the commit production is running. --against now agrees with what is served. The README count test derives its numbers from the pin instead of spelling them out, so the next roster change is a regenerate rather than an edit to a test. Co-Authored-By: Claude Opus 5 --- README.md | 4 ++-- docs/protocols/PROTOCOL-COVERAGE.json | 8 ++++---- docs/protocols/PROTOCOL-COVERAGE.md | 8 ++++---- scripts/universe/protocol-contract.test.mjs | 12 +++++++++++- 4 files changed, 21 insertions(+), 11 deletions(-) diff --git a/README.md b/README.md index 13808f48e8..3fe51e77fa 100644 --- a/README.md +++ b/README.md @@ -113,9 +113,9 @@ explorer states plainly which ones it can actually read. -6 of the 38 protocols in the registry are readable today: +7 of the 38 protocols in the registry are readable today: -- On bitcoin: **Ordinals**, **Rare Sats**, **Runes**, from ord. +- On bitcoin: **Ordinals**, **Rare Sats**, **Runes**, **OP Inscriptions**, from ord and index-opinscriptions. - On zcash: **Zerdinals**, **ZRunes**, **ZRC-20**, from index-zcash-metaprotocols. diff --git a/docs/protocols/PROTOCOL-COVERAGE.json b/docs/protocols/PROTOCOL-COVERAGE.json index e155acb089..80246c0f78 100644 --- a/docs/protocols/PROTOCOL-COVERAGE.json +++ b/docs/protocols/PROTOCOL-COVERAGE.json @@ -2,8 +2,8 @@ "schemaVersion": "universe-explorer-protocol-manifest-v1", "registryVersion": "1.0.0", "sourceRepository": "bitcoinuniverseio/backend-apis", - "sourceSha": "88a616cfdabc93facd3dcc3296f88dc214c24a40", - "generatedAt": "2026-08-30T18:18:38.168Z", + "sourceSha": "100c0ec0ccd3fd28c3f7c89bec7cb6b5575c8d96", + "generatedAt": "2026-08-30T20:52:56.287Z", "primaryStrip": [ "ORDINALS", "RUNES", @@ -400,9 +400,9 @@ "visualToken": "protocol-op-inscriptions", "implementedReadOperations": [], "authorizedReadOperations": [], - "releaseStatus": "BLOCKED", + "releaseStatus": "VERIFIED READ ONLY", "indexerAuthority": "index-opinscriptions", - "coverage": "unknown" + "coverage": "complete" }, { "schemaVersion": "universe-explorer-protocol-v1", diff --git a/docs/protocols/PROTOCOL-COVERAGE.md b/docs/protocols/PROTOCOL-COVERAGE.md index 6281aa4bd5..29954e39fd 100644 --- a/docs/protocols/PROTOCOL-COVERAGE.md +++ b/docs/protocols/PROTOCOL-COVERAGE.md @@ -20,11 +20,11 @@ A protocol never silently disappears from this table: `PROTOCOL-ROSTER.lock` records every id that has been published, and the gate fails when one of them stops appearing. -Pinned from bitcoinuniverseio/backend-apis at commit 88a616cfdabc93facd3dcc3296f88dc214c24a40, +Pinned from bitcoinuniverseio/backend-apis at commit 100c0ec0ccd3fd28c3f7c89bec7cb6b5575c8d96, manifest schema universe-explorer-protocol-manifest-v1, registry version 1.0.0, -recorded 2026-08-30T18:18:38.168Z. +recorded 2026-08-30T20:52:56.287Z. -6 of 38 protocols are readable today; the rest are recorded here but not yet served. +7 of 38 protocols are readable today; the rest are recorded here but not yet served. | id | family | chain | authority | release status | coverage | |---|---|---|---|---|---| @@ -45,7 +45,7 @@ recorded 2026-08-30T18:18:38.168Z. | arc20 | ATOMICALS | bitcoin | index-atomicals | BLOCKED | unknown | | op_return | OP DATA | bitcoin | index-op20 | BLOCKED | unknown | | op_names | OP DATA | bitcoin | index-op20 | BLOCKED | unknown | -| op_inscriptions | OP DATA | bitcoin | index-opinscriptions | BLOCKED | unknown | +| op_inscriptions | OP DATA | bitcoin | index-opinscriptions | VERIFIED READ ONLY | complete | | op_drop | OP DATA | bitcoin | index-drops-and-opdrop | BLOCKED | unknown | | drops | OP DATA | bitcoin | index-drops-and-opdrop | BLOCKED | unknown | | brc20 | OTHER | bitcoin | index-brc20 | BLOCKED | unknown | diff --git a/scripts/universe/protocol-contract.test.mjs b/scripts/universe/protocol-contract.test.mjs index a272110f19..9a7406513d 100644 --- a/scripts/universe/protocol-contract.test.mjs +++ b/scripts/universe/protocol-contract.test.mjs @@ -201,7 +201,17 @@ test('the copy reader finds the ids the frontend actually writes prose for', asy test('the README block states the roster it was generated from', async () => { const block = readmeBlockOf(await readFile(PATHS.readme, 'utf8')); assert.equal(block, renderReadmeBlock(pinned)); - assert.match(block, /6 of the 38 protocols in the registry are readable/); + // Counted from the pin rather than written down, so a roster change is a + // regenerate rather than an edit here. + const readable = pinned.protocols.filter((p) => + ['VERIFIED READ ONLY', 'PRODUCTION VERIFIED'].includes(p.releaseStatus), + ).length; + assert.match( + block, + new RegExp( + `${readable} of the ${pinned.protocols.length} protocols in the registry are readable`, + ), + ); }); test('a served roster that gained a protocol is named, not tolerated', () => { From 2f2a62d7dc96f148c6ce9144dcf9775e541800c3 Mon Sep 17 00:00:00 2001 From: Bitcoin Universe Date: Sun, 30 Aug 2026 20:02:19 +0000 Subject: [PATCH 15/15] Show every provable asset on chain transaction and address pages The transaction reading now carries each input's and output's asset inventory: protocol chips with the asset's own name (the id when no name exists), exact quantities shifted by the asset's own decimals, links to the protocol detail routes, and a named coverage state whenever emptiness was not proven. Protocol actions stop being bare labels: they carry the asset, the exact quantity, and the outpoints the action touched. Address pages gain an asset holdings panel fed by the new holdings endpoint: aggregates across unspent outputs, address-level ledger balances in their own clearly explained section so a quantity is never counted twice, an assets column on the unspent output table, the privacy boundary verbatim for shielded Zcash addresses, and honest notes when a reading is partial, unscanned, or out of an authority's coverage. Protocol copy that called Doginals and DRC-20 another chain's business is corrected, and the six chain protocols that had no copy at all get real descriptions. The visual matrix fixtures exercise a multi-asset output, a proven-empty output, an unscanned output, and both holdings views. Co-Authored-By: Claude Fable 5 --- .../multichain-explorer.component.html | 176 +++++++-- .../multichain-explorer.component.scss | 58 +++ .../multichain-explorer.component.ts | 50 ++- .../multichain-view.spec.ts | 287 +++++++++++++++ .../multichain-explorer/multichain-view.ts | 347 +++++++++++++++++- .../src/app/universe/universe-api.service.ts | 19 + .../app/universe/universe-protocol-copy.ts | 24 +- scripts/universe/visual-qa/chain-fixtures.mjs | 195 +++++++++- 8 files changed, 1108 insertions(+), 48 deletions(-) diff --git a/frontend/src/app/universe/multichain-explorer/multichain-explorer.component.html b/frontend/src/app/universe/multichain-explorer/multichain-explorer.component.html index e0a07e1684..a49c37242b 100644 --- a/frontend/src/app/universe/multichain-explorer/multichain-explorer.component.html +++ b/frontend/src/app/universe/multichain-explorer/multichain-explorer.component.html @@ -1,5 +1,27 @@
    + + +
    + + {{ chipRow.protocolLabel }} + {{ chipRow.name.length > 24 ? short(chipRow.name, 8) : chipRow.name }} + + {{ chipRow.name.length > 24 ? short(chipRow.name, 8) : chipRow.name }} + + + {{ chipRow.quantity.display }} {{ chipRow.ticker }} + + + {{ assets.coverageLabel }} +
    +     +
    @@ -196,6 +218,7 @@

    Why this tran

    Transparent value

    +

    @@ -204,16 +227,19 @@

    1. - - {{ short(party.address, 12) }} - - Newly issued by the block - No standard address - - - {{ party.amount.display }} - not reported +
      + + {{ short(party.address, 12) }} + + Newly issued by the block + No standard address + + + {{ party.amount.display }} + not reported +
      +

    @@ -230,14 +256,21 @@

    1. - - {{ short(party.address, 12) }} - No standard address - Spent - - {{ party.amount.display }} - not reported +
      + + {{ short(party.address, 12) }} + No standard address + #{{ party.index }} + Spent + + {{ party.amount.display }} + not reported +
      +
    @@ -265,13 +298,29 @@

    Shielded st aria-labelledby="tx-actions-heading">

    Protocol actions

    + + {{ action.protocolLabel }} + {{ action.actionType }} + + {{ action.assetName.length > 24 ? short(action.assetName, 8) : action.assetName }} + + {{ action.assetName.length > 24 ? short(action.assetName, 8) : action.assetName }} + + + + {{ action.quantity.display }} {{ action.assetTicker }} + + {{ short(op, 6) }} + {{ action.stateLabel }} + +

    Evaluated

    • - {{ action.protocolLabel }} - {{ action.actionType }} - {{ action.stateLabel }} +
    @@ -283,9 +332,7 @@

    Candidates

    • - {{ action.protocolLabel }} - {{ action.actionType }} - {{ action.stateLabel }} +
    @@ -392,6 +439,83 @@

    Balance

    +
    +

    Asset holdings

    + +

    {{ hold.privacyNotice }}

    +

    {{ hold.completenessNote }}

    + + +

    Held on unspent outputs

    +
    + + + + + + + + + + + + + + + + + +
    ProtocolAssetQuantityOutputs
    {{ row.protocolLabel }} + {{ row.name.length > 24 ? short(row.name, 8) : row.name }} + {{ row.name.length > 24 ? short(row.name, 8) : row.name }} + {{ row.ticker }} + + {{ row.quantity.display }} + not summable + {{ row.utxoCount?.display }}
    +
    +     + + +

    Address-level balances

    +

    + These protocols keep their ledger per address rather than per output, so these balances belong to the address as a whole and are never assigned to any single unspent output above. +

    +
    + + + + + + + + + + + + + + + + + + + +
    ProtocolAssetBalanceAvailableTransferable
    {{ row.protocolLabel }} + {{ row.name.length > 24 ? short(row.name, 8) : row.name }} + {{ row.name.length > 24 ? short(row.name, 8) : row.name }} + {{ row.quantity?.display }}{{ row.available?.display }}{{ row.transferable?.display }}
    +
    +     + +

    + Every unspent output was checked against the protocol authorities and none carries a protocol asset. +

    +
    +

    Unspent outputs

    @@ -401,6 +525,7 @@

    Unspent o Outpoint Amount Block + Assets @@ -421,6 +546,9 @@

    Unspent o i18n="universe.chain.utxo-height-unstated">not reported + + + diff --git a/frontend/src/app/universe/multichain-explorer/multichain-explorer.component.scss b/frontend/src/app/universe/multichain-explorer/multichain-explorer.component.scss index c46cc5e524..35b45959e0 100644 --- a/frontend/src/app/universe/multichain-explorer/multichain-explorer.component.scss +++ b/frontend/src/app/universe/multichain-explorer/multichain-explorer.component.scss @@ -824,3 +824,61 @@ margin-top: var(--u-space-2); } } + +/* ------------------------------------------------------------------------ */ +/* Per-outpoint asset chips and the address holdings panel */ +/* ------------------------------------------------------------------------ */ + +/* A party row grew a second line for its assets, so the two-column grid + moved from the list item onto the first line. */ +.party-list li { + display: block; +} + +.party-main { + display: grid; + grid-template-columns: minmax(0, 1fr) max-content; + align-items: baseline; + gap: var(--u-space-3); +} + +.party-assets { + display: flex; + flex-wrap: wrap; + align-items: center; + gap: var(--u-space-2); + margin-top: var(--u-space-1); +} + +.asset-chip { + display: inline-flex; + align-items: baseline; + gap: var(--u-space-1); + min-width: 0; +} + +.asset-name { + overflow-wrap: anywhere; +} + +.asset-quantity, +.action-quantity { + font-family: var(--u-font-data); + font-variant-numeric: tabular-nums; +} + +.action-asset { + overflow-wrap: anywhere; +} + +.assets-cell { + max-width: 28rem; +} + +.assets-cell .party-assets { + margin-top: 0; +} + +.holdings h3 { + margin-top: var(--u-space-4); +} diff --git a/frontend/src/app/universe/multichain-explorer/multichain-explorer.component.ts b/frontend/src/app/universe/multichain-explorer/multichain-explorer.component.ts index c6fd6e069e..2c64e10b0b 100644 --- a/frontend/src/app/universe/multichain-explorer/multichain-explorer.component.ts +++ b/frontend/src/app/universe/multichain-explorer/multichain-explorer.component.ts @@ -52,12 +52,15 @@ import { readNotReadyReasons, readOutpoint, readProtocolCoverage, + readAddressHoldings, readRecordFacts, readSourceDetails, readStatusRail, readTransaction, readTransactionList, shortenIdentifier, + AddressHoldingsReading, + OutpointAssetsReading, } from '@app/universe/multichain-explorer/multichain-view'; import { ChainReasonReading } from '@app/universe/multichain-explorer/chain-reasons'; import { @@ -125,6 +128,8 @@ interface ExplorerViewModel { readonly transaction: TransactionReading | null; readonly block: BlockReading | null; readonly address: AddressReading | null; + /** The address's protocol asset holdings, served beside the base view. */ + readonly holdings: AddressHoldingsReading | null; readonly outpoint: OutpointReading | null; readonly collection: CollectionReading | null; readonly transactionList: TransactionListReading | null; @@ -183,6 +188,8 @@ const PRESENTED_FIELDS: Partial> = { 'transparent', 'shielded', 'protocolActions', 'replacement', 'expiry', 'confirmationsAtomic', 'sizeBytesAtomic', 'virtualSizeBytesAtomic', 'firstSeenAt', 'completeness', + // The per-outpoint asset readings own this summary block. + 'assetFlow', ], block: [ 'chain', 'network', 'block', 'pagination', @@ -295,15 +302,29 @@ export class MultichainExplorerComponent implements OnInit, OnDestroy { catchError(() => of(null)) ) : of(null), + // The asset holdings beside the address view. Same rule: a + // protocol authority that cannot answer costs the asset + // sections, never the address page. + context.page === 'address' && this.reference + ? this.api + .getChainAddressHoldings$( + this.chain, + this.reference, + LIST_PAGE_SIZE, + (context.listPage - 1) * LIST_PAGE_SIZE + ) + .pipe(catchError(() => of(null))) + : of(null), ]) ), - map(([status, result, bucketsPayload]): ExplorerViewModel => + map(([status, result, bucketsPayload, holdingsPayload]): ExplorerViewModel => this.viewModel( status.capability, status.error, result.payload, result.error, - bucketsPayload + bucketsPayload, + holdingsPayload ) ) ); @@ -322,7 +343,8 @@ export class MultichainExplorerComponent implements OnInit, OnDestroy { capabilityError: string | null, payload: ChainExplorerPayload | null, payloadError: string | null, - bucketsPayload: ChainExplorerPayload | null = null + bucketsPayload: ChainExplorerPayload | null = null, + holdingsPayload: ChainExplorerPayload | null = null ): ExplorerViewModel { const shape = classifyPayload(payload); const transaction = payload && shape === 'transaction' ? readTransaction(payload, this.profile) : null; @@ -387,6 +409,9 @@ export class MultichainExplorerComponent implements OnInit, OnDestroy { transaction, block, address, + holdings: holdingsPayload + ? readAddressHoldings(holdingsPayload, this.profile) + : null, outpoint, collection, transactionList, @@ -485,6 +510,25 @@ export class MultichainExplorerComponent implements OnInit, OnDestroy { return this.route.snapshot.paramMap.get('protocol') ?? ''; } + /** + * The asset reading for one base-view unspent output, joined from the + * holdings view by outpoint. Null when the holdings page does not cover + * this output, which renders as no claim rather than as empty. + */ + holdingsAssetsFor( + holdings: AddressHoldingsReading | null, + txid: string, + vout: string + ): OutpointAssetsReading | null { + if (!holdings) { + return null; + } + return ( + holdings.utxos.find((utxo) => utxo.txid === txid && utxo.vout === vout) + ?.assets ?? null + ); + } + short(value: string, lead = 8): string { return shortenIdentifier(value, lead); } diff --git a/frontend/src/app/universe/multichain-explorer/multichain-view.spec.ts b/frontend/src/app/universe/multichain-explorer/multichain-view.spec.ts index 33530a4f62..e4162a4716 100644 --- a/frontend/src/app/universe/multichain-explorer/multichain-view.spec.ts +++ b/frontend/src/app/universe/multichain-explorer/multichain-view.spec.ts @@ -19,6 +19,7 @@ import { readEmptyList, readHistoryCoverage, readNotReadyReasons, + readAddressHoldings, readOutpoint, readPaging, readProtocolCoverage, @@ -932,3 +933,289 @@ describe('readProtocolCoverage reasons', () => { expect(drc20?.reasons[0].text).not.toContain('-disabled'); }); }); + +describe('per-outpoint asset readings', () => { + const duneTx = (): ChainExplorerPayload => ({ + schemaVersion: 'universe-transaction-v1', + txid: TXID, + status: 'confirmed', + transparent: { + inputs: [ + { + indexAtomic: '0', + previousOutpoint: `${'c'.repeat(64)}:1`, + address: 'D8input', + valueAtomic: '100000000', + coinbase: false, + assets: { positions: [], coverage: 'out-of-coverage', coveredProtocolIds: [] }, + }, + ], + outputs: [ + { + indexAtomic: '0', + address: 'D8output', + valueAtomic: '100000', + spent: false, + assets: { + positions: [ + { + outpoint: `${TXID}:0`, + vout: 0, + valueSatsAtomic: '100000', + asset: { + protocolId: 'dunes', + assetId: 'WOW SUCH DUNE', + displayName: 'WOW SUCH DUNE', + ticker: 'W', + assetKind: 'fungible', + decimals: 8, + }, + quantityAtomic: '150000000', + state: 'active', + }, + { + outpoint: `${TXID}:0`, + vout: 0, + valueSatsAtomic: '100000', + asset: { + protocolId: 'doginals', + assetId: `${'d'.repeat(64)}i0`, + assetKind: 'inscription', + }, + state: 'active', + }, + ], + coverage: 'complete', + coveredProtocolIds: ['doginals', 'dunes'], + }, + }, + { + indexAtomic: '1', + address: 'D8empty', + valueAtomic: '5000', + spent: false, + assets: { positions: [], coverage: 'proven-empty', coveredProtocolIds: ['doginals', 'dunes'] }, + }, + ], + }, + protocolActions: { + candidates: [], + confirmed: [ + { + eventId: 'e1', + protocolId: 'dunes', + state: 'confirmed-accepted', + actionType: 'etch', + evidenceIds: [], + asset: { + protocolId: 'dunes', + assetId: 'WOW SUCH DUNE', + displayName: 'WOW SUCH DUNE', + ticker: 'W', + assetKind: 'fungible', + decimals: 8, + }, + quantityAtomic: '150000000', + outputOutpoints: [`${TXID}:0`], + }, + ], + }, + }); + + it('reads asset chips on outputs with quantities shifted by the asset decimals', () => { + const reading = readTransaction(duneTx(), DOGE); + const output = reading?.outputs[0]; + expect(output?.outpoint).toBe(`${TXID}:0`); + const dune = output?.assets?.chips[0]; + expect(dune?.protocolLabel).toBe('Dunes'); + expect(dune?.name).toBe('WOW SUCH DUNE'); + // 150000000 shifted by the dune's own 8 decimals, not the chain's. + expect(dune?.quantity?.display).toBe('1.5'); + expect(dune?.quantity?.exact).toBe('150000000'); + expect(dune?.link).toEqual(['/', 'dogecoin', 'protocols', 'dunes', 'WOW SUCH DUNE']); + const doginal = output?.assets?.chips[1]; + expect(doginal?.protocolLabel).toBe('Doginals'); + // The asset id is the mandated fallback name, never a blank label. + expect(doginal?.name).toBe(`${'d'.repeat(64)}i0`); + expect(doginal?.quantity).toBeNull(); + }); + + it('keeps proven emptiness and out-of-coverage distinguishable', () => { + const reading = readTransaction(duneTx(), DOGE); + const empty = reading?.outputs[1].assets; + expect(empty?.provenEmpty).toBe(true); + expect(empty?.coverageLabel).toBeNull(); + const input = reading?.inputs[0].assets; + expect(input?.provenEmpty).toBe(false); + expect(input?.coverageLabel).not.toBeNull(); + expect(input?.coverageTone).toBe('neutral'); + }); + + it('carries asset identity, exact quantity and outpoints on actions', () => { + const reading = readTransaction(duneTx(), DOGE); + const action = reading?.confirmedActions[0]; + expect(action?.assetName).toBe('WOW SUCH DUNE'); + expect(action?.assetTicker).toBe('W'); + expect(action?.quantity?.display).toBe('1.5'); + expect(action?.quantity?.exact).toBe('150000000'); + expect(action?.outputOutpoints).toEqual([`${TXID}:0`]); + expect(action?.assetLink).toEqual(['/', 'dogecoin', 'protocols', 'dunes', 'WOW SUCH DUNE']); + }); + + it('reads actions without assets exactly as before', () => { + const payload = duneTx(); + (payload.protocolActions as Record).confirmed = [ + { eventId: 'e2', protocolId: 'drc20', state: 'confirmed-rejected', actionType: 'transfer', evidenceIds: [] }, + ]; + const action = readTransaction(payload, DOGE)?.confirmedActions[0]; + expect(action?.assetName).toBeNull(); + expect(action?.quantity).toBeNull(); + expect(action?.inputOutpoints).toEqual([]); + }); +}); + +describe('readAddressHoldings', () => { + const view = (overrides: Record = {}): ChainExplorerPayload => ({ + schemaVersion: 'universe-address-holdings-v1', + chain: 'zcash', + network: 'mainnet', + address: 't1Example', + utxos: [ + { + outpoint: `${TXID}:0`, + txid: TXID, + vout: 0, + valueAtomic: '250000000', + assets: { + positions: [ + { + outpoint: `${TXID}:0`, + vout: 0, + valueSatsAtomic: '250000000', + asset: { + protocolId: 'zrunes', + assetId: '10:1', + displayName: 'ZRUNE ONE', + ticker: 'Z', + assetKind: 'fungible', + decimals: 2, + }, + quantityAtomic: '12345', + state: 'active', + }, + ], + coverage: 'complete', + coveredProtocolIds: ['zerdinals', 'zrunes'], + }, + }, + ], + addressLevelBalances: [ + { + asset: { + protocolId: 'zrc20', + assetId: 'zord:zero', + ticker: 'ZERO', + assetKind: 'fungible', + decimals: 8, + }, + quantityAtomic: '900000000', + availableAtomic: '400000000', + transferableAtomic: '500000000', + decimals: 8, + semantics: 'zrc20-zord-ledger', + }, + ], + aggregateHoldings: [ + { + asset: { + protocolId: 'zrunes', + assetId: '10:1', + displayName: 'ZRUNE ONE', + ticker: 'Z', + assetKind: 'fungible', + decimals: 2, + }, + quantityAtomic: '12345', + utxoCountAtomic: '1', + source: 'utxo-bound', + }, + ], + paging: { + limitAtomic: '50', + offsetAtomic: '0', + returnedAtomic: '1', + totalUtxoCountAtomic: '1', + hasMore: false, + }, + checkpoint: { chain: 'zcash', network: 'mainnet', heightAtomic: '3131000', blockHash: BLOCK_HASH, reorgEpoch: '0', observedAt: '2026-08-30T00:00:00.000Z' }, + sourceEvidence: [], + complete: true, + unknownAttachmentCount: 0, + outOfCoverageCount: 0, + ...overrides, + }); + + it('reads the complete holdings view exactly', () => { + const reading = readAddressHoldings(view(), ZEC); + expect(reading?.address).toBe('t1Example'); + expect(reading?.utxos[0].outpoint).toBe(`${TXID}:0`); + expect(reading?.utxos[0].amount?.display).toBe('2.5'); + expect(reading?.utxos[0].assets?.chips[0].quantity?.display).toBe('123.45'); + expect(reading?.aggregates[0].quantity?.exact).toBe('12345'); + expect(reading?.aggregates[0].utxoCount?.display).toBe('1'); + expect(reading?.addressLevel[0].quantity?.display).toBe('9'); + expect(reading?.addressLevel[0].available?.display).toBe('4'); + expect(reading?.addressLevel[0].transferable?.display).toBe('5'); + expect(reading?.complete).toBe(true); + expect(reading?.completenessNote).toBeNull(); + expect(reading?.privacyNotice).toBeNull(); + expect(reading?.checkpointHeight?.exact).toBe('3131000'); + }); + + it('never renders a partial reading as complete', () => { + const reading = readAddressHoldings( + view({ complete: false, unknownAttachmentCount: 2 }), + ZEC + ); + expect(reading?.complete).toBe(false); + expect(reading?.completenessNote).not.toBeNull(); + }); + + it('passes the privacy boundary through verbatim instead of a fabricated empty account', () => { + const reading = readAddressHoldings( + view({ + utxos: [], + aggregateHoldings: [], + addressLevelBalances: [], + privacy: { publiclyObservable: false, notice: 'Shielded activity is not publicly observable.' }, + }), + ZEC + ); + expect(reading?.privacyNotice).toBe('Shielded activity is not publicly observable.'); + expect(reading?.utxos).toEqual([]); + expect(reading?.completenessNote).toBeNull(); + }); + + it('refuses payloads that are not the holdings contract', () => { + expect(readAddressHoldings({ schemaVersion: 'other', address: 'x' }, ZEC)).toBeNull(); + expect(readAddressHoldings(null, ZEC)).toBeNull(); + }); + + it('never publishes a partial aggregate sum', () => { + const reading = readAddressHoldings( + view({ + aggregateHoldings: [ + { + asset: { protocolId: 'zrunes', assetId: '10:1', assetKind: 'fungible' }, + quantityAtomic: null, + utxoCountAtomic: '2', + source: 'utxo-bound', + }, + ], + }), + ZEC + ); + expect(reading?.aggregates[0].quantity).toBeNull(); + expect(reading?.aggregates[0].utxoCount?.exact).toBe('2'); + }); +}); diff --git a/frontend/src/app/universe/multichain-explorer/multichain-view.ts b/frontend/src/app/universe/multichain-explorer/multichain-view.ts index e740137edf..44e4aa5e6c 100644 --- a/frontend/src/app/universe/multichain-explorer/multichain-view.ts +++ b/frontend/src/app/universe/multichain-explorer/multichain-view.ts @@ -1105,6 +1105,118 @@ export function classifyPayload(payload: ChainExplorerPayload | null): ChainShap return 'record'; } +/** + * One protocol asset attached to an outpoint, ready to render: protocol + * label from the chain profile, the asset's own name with the id as the + * mandated fallback, and the exact quantity shifted by the asset's own + * decimals rather than the chain's precision. + */ +export interface AssetChipRow { + readonly protocolId: string; + readonly protocolLabel: string; + readonly name: string; + readonly assetId: string; + readonly ticker: string | null; + readonly quantity: ExactNumber | null; + readonly kind: string; + /** Route to the asset's detail page, when this explorer serves one. */ + readonly link: readonly string[] | null; +} + +/** + * The asset reading for one outpoint. An empty chip list means nothing by + * itself: `coverageLabel` says whether emptiness was proven, unscanned, + * outside the authority's coverage, or simply unavailable, and only a + * proven empty renders as the quiet compact row. + */ +export interface OutpointAssetsReading { + readonly chips: readonly AssetChipRow[]; + readonly coverageLabel: string | null; + readonly coverageTone: EvidenceTone; + readonly provenEmpty: boolean; +} + +const ASSET_COVERAGE_LABEL: Record = { + partial: $localize`:@@universe.chain.assets-partial:Partially checked`, + unscanned: $localize`:@@universe.chain.assets-unscanned:Not scanned yet`, + 'out-of-coverage': $localize`:@@universe.chain.assets-out-of-coverage:Outside the authority's coverage`, + unavailable: $localize`:@@universe.chain.assets-unavailable:Asset authority unavailable`, +}; + +const ASSET_COVERAGE_TONE: Record = { + complete: 'proven', + 'proven-empty': 'proven', + partial: 'partial', + unscanned: 'neutral', + 'out-of-coverage': 'neutral', + unavailable: 'unavailable', +}; + +function protocolTabFor( + profile: ChainProfile, + protocolId: string +): ChainProtocolTab | null { + return ( + profile.protocols.find( + (tab) => tab.id === protocolId || tab.registryIds.includes(protocolId) + ) ?? null + ); +} + +function assetChip(entry: Record, profile: ChainProfile): AssetChipRow | null { + const asset = isRecord(entry.asset) ? entry.asset : null; + if (!asset) { + return null; + } + const protocolId = text(asset.protocolId) ?? ''; + const assetId = text(asset.assetId) ?? ''; + if (!protocolId || !assetId) { + return null; + } + const tab = protocolTabFor(profile, protocolId); + const decimals = + typeof asset.decimals === 'number' && + Number.isSafeInteger(asset.decimals) && + asset.decimals >= 0 && + asset.decimals <= 77 + ? asset.decimals + : 0; + return { + protocolId, + protocolLabel: tab?.label ?? humanizeFieldName(protocolId), + // A missing display name falls back to the asset id, never to a blank. + name: text(asset.displayName) ?? assetId, + assetId, + ticker: text(asset.ticker), + quantity: formatAtomicAmount(text(entry.quantityAtomic), decimals), + kind: text(asset.assetKind) ?? 'unknown', + link: tab + ? ['/', profile.chain, 'protocols', tab.id, assetId] + : null, + }; +} + +/** Reads the additive per-outpoint `assets` field a transaction view carries. */ +export function readOutpointAssets( + value: unknown, + profile: ChainProfile +): OutpointAssetsReading | null { + if (!isRecord(value) || !Array.isArray(value.positions)) { + return null; + } + const chips = value.positions + .filter(isRecord) + .map((entry) => assetChip(entry, profile)) + .filter((chip): chip is AssetChipRow => chip !== null); + const coverage = text(value.coverage) ?? ''; + return { + chips, + coverageLabel: ASSET_COVERAGE_LABEL[coverage] ?? null, + coverageTone: ASSET_COVERAGE_TONE[coverage] ?? 'neutral', + provenEmpty: coverage === 'proven-empty' && chips.length === 0, + }; +} + export interface TransactionPartyRow { readonly index: string; readonly address: string | null; @@ -1112,6 +1224,9 @@ export interface TransactionPartyRow { readonly reference: string | null; readonly coinbase: boolean; readonly spent: boolean | null; + /** The outpoint this row is, when it can be stated. */ + readonly outpoint: string | null; + readonly assets: OutpointAssetsReading | null; } export interface ShieldedReading { @@ -1127,6 +1242,15 @@ export interface ProtocolActionRow { readonly actionType: string; readonly stateLabel: string; readonly tone: EvidenceTone; + /** The asset the action concerns, when the authority named one. */ + readonly assetName: string | null; + readonly assetTicker: string | null; + readonly assetLink: readonly string[] | null; + /** Exact quantity in the asset's atomic unit, shifted by its decimals. */ + readonly quantity: ExactNumber | null; + /** Outpoints the action consumed and produced, for linking. */ + readonly inputOutpoints: readonly string[]; + readonly outputOutpoints: readonly string[]; } export interface TransactionReading { @@ -1225,19 +1349,41 @@ function sumAtomic( function partyRows( values: unknown, profile: ChainProfile, - kind: 'input' | 'output' + kind: 'input' | 'output', + txid: string ): readonly TransactionPartyRow[] { if (!Array.isArray(values)) { return []; } - return values.filter(isRecord).map((entry) => ({ - index: text(entry.indexAtomic) ?? '', - address: text(entry.address), - amount: formatAtomicAmount(text(entry.valueAtomic), profile.precision), - reference: kind === 'input' ? text(entry.previousOutpoint) : null, - coinbase: entry.coinbase === true, - spent: typeof entry.spent === 'boolean' ? entry.spent : null, - })); + return values.filter(isRecord).map((entry) => { + const index = text(entry.indexAtomic) ?? ''; + const reference = kind === 'input' ? text(entry.previousOutpoint) : null; + return { + index, + address: text(entry.address), + amount: formatAtomicAmount(text(entry.valueAtomic), profile.precision), + reference, + coinbase: entry.coinbase === true, + spent: typeof entry.spent === 'boolean' ? entry.spent : null, + outpoint: + kind === 'output' + ? txid && index !== '' + ? `${txid}:${index}` + : null + : reference, + assets: readOutpointAssets(entry.assets, profile), + }; + }); +} + +function outpointList(value: unknown): readonly string[] { + if (!Array.isArray(value)) { + return []; + } + return value.filter( + (entry): entry is string => + typeof entry === 'string' && /^[0-9a-f]{64}:\d{1,10}$/.test(entry) + ); } function actionRows( @@ -1250,15 +1396,33 @@ function actionRows( return values.filter(isRecord).map((entry) => { const state = text(entry.state) ?? ''; const protocolId = text(entry.protocolId) ?? ''; + const asset = isRecord(entry.asset) ? entry.asset : null; + const assetId = asset ? text(asset.assetId) : null; + const tab = protocolTabFor(profile, protocolId); + const decimals = + asset && + typeof asset.decimals === 'number' && + Number.isSafeInteger(asset.decimals) && + asset.decimals >= 0 && + asset.decimals <= 77 + ? asset.decimals + : 0; return { eventId: text(entry.eventId) ?? '', protocolId, - protocolLabel: - profile.protocols.find((tab) => tab.id === protocolId)?.label ?? - humanizeFieldName(protocolId), + protocolLabel: tab?.label ?? humanizeFieldName(protocolId), actionType: humanizeFieldName(text(entry.actionType) ?? ''), stateLabel: PROTOCOL_STATE_LABEL[state] ?? availabilityLabel(state), tone: PROTOCOL_STATE_TONE[state] ?? 'neutral', + assetName: asset ? (text(asset.displayName) ?? assetId) : null, + assetTicker: asset ? text(asset.ticker) : null, + assetLink: + tab && assetId + ? ['/', profile.chain, 'protocols', tab.id, assetId] + : null, + quantity: formatAtomicAmount(text(entry.quantityAtomic), decimals), + inputOutpoints: outpointList(entry.inputOutpoints), + outputOutpoints: outpointList(entry.outputOutpoints), }; }); } @@ -1291,9 +1455,10 @@ export function readTransaction( return null; } const status = text(payload.status) ?? ''; + const txid = text(payload.txid) ?? ''; const transparent = isRecord(payload.transparent) ? payload.transparent : {}; - const inputs = partyRows(transparent.inputs, profile, 'input'); - const outputs = partyRows(transparent.outputs, profile, 'output'); + const inputs = partyRows(transparent.inputs, profile, 'input', txid); + const outputs = partyRows(transparent.outputs, profile, 'output', txid); const fee = isRecord(payload.fee) ? payload.fee : {}; const block = isRecord(payload.block) ? payload.block : null; const replacement = isRecord(payload.replacement) ? payload.replacement : null; @@ -1642,6 +1807,160 @@ export function readAddress( }; } +/** One unspent output from the holdings view, with its asset reading. */ +export interface EnrichedUtxoRow { + readonly txid: string; + readonly vout: string; + readonly outpoint: string; + readonly amount: ExactNumber | null; + readonly height: ExactNumber | null; + readonly assets: OutpointAssetsReading | null; +} + +/** One asset holding row: an aggregate or an address-level balance. */ +export interface HoldingRow { + readonly protocolId: string; + readonly protocolLabel: string; + readonly name: string; + readonly assetId: string; + readonly ticker: string | null; + readonly quantity: ExactNumber | null; + readonly utxoCount: ExactNumber | null; + readonly available: ExactNumber | null; + readonly transferable: ExactNumber | null; + readonly link: readonly string[] | null; +} + +/** + * The address asset-holdings view: the paginated unspent outputs with + * their attached assets, aggregates by asset, and the balances the + * protocol defines at address level, kept apart so the same quantity is + * never counted in both sections. + */ +export interface AddressHoldingsReading { + readonly address: string; + readonly utxos: readonly EnrichedUtxoRow[]; + readonly aggregates: readonly HoldingRow[]; + readonly addressLevel: readonly HoldingRow[]; + readonly hasMore: boolean; + readonly totalUtxos: ExactNumber | null; + readonly complete: boolean; + /** Set when the reading is incomplete; says what kind of hole exists. */ + readonly completenessNote: string | null; + readonly privacyNotice: string | null; + readonly checkpointHeight: ExactNumber | null; +} + +function holdingRow( + entry: Record, + profile: ChainProfile +): HoldingRow | null { + const asset = isRecord(entry.asset) ? entry.asset : null; + if (!asset) { + return null; + } + const protocolId = text(asset.protocolId) ?? ''; + const assetId = text(asset.assetId) ?? ''; + if (!protocolId || !assetId) { + return null; + } + const tab = protocolTabFor(profile, protocolId); + const decimals = + typeof asset.decimals === 'number' && + Number.isSafeInteger(asset.decimals) && + asset.decimals >= 0 && + asset.decimals <= 77 + ? asset.decimals + : 0; + return { + protocolId, + protocolLabel: tab?.label ?? humanizeFieldName(protocolId), + name: text(asset.displayName) ?? assetId, + assetId, + ticker: text(asset.ticker), + quantity: formatAtomicAmount(text(entry.quantityAtomic), decimals), + utxoCount: formatExactInteger(text(entry.utxoCountAtomic)), + available: formatAtomicAmount(text(entry.availableAtomic), decimals), + transferable: formatAtomicAmount(text(entry.transferableAtomic), decimals), + link: tab ? ['/', profile.chain, 'protocols', tab.id, assetId] : null, + }; +} + +/** Reads the address-holdings endpoint's `universe-address-holdings-v1` view. */ +export function readAddressHoldings( + payload: unknown, + profile: ChainProfile +): AddressHoldingsReading | null { + if ( + !isRecord(payload) || + payload.schemaVersion !== 'universe-address-holdings-v1' + ) { + return null; + } + const address = text(payload.address); + if (!address) { + return null; + } + const utxos = Array.isArray(payload.utxos) + ? payload.utxos.filter(isRecord) + : []; + const paging = isRecord(payload.paging) ? payload.paging : {}; + const privacy = isRecord(payload.privacy) ? payload.privacy : null; + const checkpoint = isRecord(payload.checkpoint) ? payload.checkpoint : null; + const rows = (value: unknown): readonly HoldingRow[] => + Array.isArray(value) + ? value + .filter(isRecord) + .map((entry) => holdingRow(entry, profile)) + .filter((row): row is HoldingRow => row !== null) + : []; + const unknownCount = Number(text(payload.unknownAttachmentCount) ?? payload.unknownAttachmentCount ?? 0); + const outOfCoverageCount = Number(text(payload.outOfCoverageCount) ?? payload.outOfCoverageCount ?? 0); + const complete = payload.complete === true; + let completenessNote: string | null = null; + if (!complete && privacy?.publiclyObservable !== false) { + if (Number.isSafeInteger(unknownCount) && unknownCount > 0) { + completenessNote = $localize`:@@universe.chain.holdings-unknown:Some outputs could not be checked against every protocol authority.`; + } else if (Number.isSafeInteger(outOfCoverageCount) && outOfCoverageCount > 0) { + completenessNote = $localize`:@@universe.chain.holdings-out-of-coverage:Some outputs are outside the protocol authorities' coverage.`; + } else { + completenessNote = $localize`:@@universe.chain.holdings-partial:This reading is not complete.`; + } + } + return { + address, + utxos: utxos.map((entry) => { + const txid = text(entry.txid) ?? ''; + const vout = + typeof entry.vout === 'number' && Number.isSafeInteger(entry.vout) + ? String(entry.vout) + : (text(entry.vout) ?? ''); + return { + txid, + vout, + outpoint: text(entry.outpoint) ?? `${txid}:${vout}`, + amount: formatAtomicAmount(text(entry.valueAtomic), profile.precision), + height: formatExactInteger(text(entry.heightAtomic)), + assets: readOutpointAssets(entry.assets, profile), + }; + }), + aggregates: rows(payload.aggregateHoldings), + addressLevel: rows(payload.addressLevelBalances), + hasMore: paging.hasMore === true, + totalUtxos: formatExactInteger(text(paging.totalUtxoCountAtomic)), + complete, + completenessNote, + privacyNotice: + privacy && privacy.publiclyObservable === false + ? (text(privacy.notice) ?? + $localize`:@@universe.chain.holdings-privacy:This address's holdings are not publicly observable.`) + : null, + checkpointHeight: checkpoint + ? formatExactInteger(text(checkpoint.heightAtomic)) + : null, + }; +} + export interface OutpointReading { readonly txid: string; readonly vout: string; diff --git a/frontend/src/app/universe/universe-api.service.ts b/frontend/src/app/universe/universe-api.service.ts index d159086a83..6a610bf51d 100644 --- a/frontend/src/app/universe/universe-api.service.ts +++ b/frontend/src/app/universe/universe-api.service.ts @@ -253,6 +253,25 @@ export class UniverseApiService { ); } + /** + * The address asset-holdings view for a chain: every paginated unspent + * output with its attached protocol assets, address-level balances, and + * exact aggregates. Served beside the base address view so a failure here + * degrades the asset sections without taking the address page down. + */ + getChainAddressHoldings$(chain: Exclude, address: string, limit = 50, offset = 0): Observable { + return this.httpClient.get( + this.apiBaseUrl + '/api/v1/' + chain + '/address/' + encodeURIComponent(address) + '/holdings?network=mainnet&limit=' + limit + '&offset=' + offset + ); + } + + /** The Bitcoin address asset-holdings view from the universe overlay. */ + getAddressHoldings$(address: string, limit = 100, offset = 0): Observable { + return this.httpClient.get( + this.apiBaseUrl + '/api/v1/universe/addresses/' + encodeURIComponent(address) + '/holdings?limit=' + limit + '&offset=' + offset + ); + } + getChainOutpoint$(chain: Exclude, txid: string, vout: string): Observable { return this.httpClient.get( this.apiBaseUrl + '/api/v1/' + chain + '/outpoint/' + encodeURIComponent(txid) + '/' + encodeURIComponent(vout) + '?network=mainnet' diff --git a/frontend/src/app/universe/universe-protocol-copy.ts b/frontend/src/app/universe/universe-protocol-copy.ts index d0bcd2a1ba..4ba7c7ca57 100644 --- a/frontend/src/app/universe/universe-protocol-copy.ts +++ b/frontend/src/app/universe/universe-protocol-copy.ts @@ -109,11 +109,31 @@ const PROTOCOL_COPY: Record = { actions: ['deploy', 'mint', 'transfer'], }, doginals: { - summary: $localize`:@@universe.copy.doginals:Doginals apply ordinal theory to Dogecoin. They are indexed on a different chain than the one this explorer serves.`, + summary: $localize`:@@universe.copy.doginals:Doginals apply ordinal theory to Dogecoin: content inscribed on individual koinu and carried by the unspent output that holds them.`, actions: ['inscribe', 'transfer'], }, drc20: { - summary: $localize`:@@universe.copy.drc20:DRC-20 is the Dogecoin counterpart of inscription-based fungible tokens, indexed on a different chain than the one this explorer serves.`, + summary: $localize`:@@universe.copy.drc20:DRC-20 is the Dogecoin counterpart of inscription-based fungible tokens. Balances belong to an address; a transfer inscription moves an amount onto one unspent output until it is spent.`, + actions: ['deploy', 'mint', 'transfer'], + }, + dunes: { + summary: $localize`:@@universe.copy.dunes:Dunes are Dogecoin's runestone-style fungible tokens: balances live directly on unspent outputs and move by edicts in the spending transaction.`, + actions: ['etch', 'mint', 'transfer'], + }, + tap_doge: { + summary: $localize`:@@universe.copy.tap-doge:TAP on Dogecoin keeps token balances in an address-level ledger derived from inscriptions, so holdings belong to the address rather than to any single output.`, + actions: ['deploy', 'mint', 'transfer'], + }, + zerdinals: { + summary: $localize`:@@universe.copy.zerdinals:Zerdinals inscribe content into Zcash transparent transactions, and each inscription is carried by the unspent output that currently holds it.`, + actions: ['inscribe', 'transfer'], + }, + zrunes: { + summary: $localize`:@@universe.copy.zrunes:ZRunes are Zcash's runestone-style fungible tokens: balances live on transparent unspent outputs and move by edicts in the spending transaction.`, + actions: ['etch', 'mint', 'transfer'], + }, + zrc20: { + summary: $localize`:@@universe.copy.zrc20:ZRC-20 tokens keep an address-level ledger on Zcash, read under two published rulesets. Where the rulesets disagree this explorer shows both readings.`, actions: ['deploy', 'mint', 'transfer'], }, }; diff --git a/scripts/universe/visual-qa/chain-fixtures.mjs b/scripts/universe/visual-qa/chain-fixtures.mjs index 2a8e9ee769..e7bda606b8 100644 --- a/scripts/universe/visual-qa/chain-fixtures.mjs +++ b/scripts/universe/visual-qa/chain-fixtures.mjs @@ -117,24 +117,69 @@ function dogecoinTransaction() { expiry: null, transparent: { inputs: [ - { indexAtomic: '0', previousOutpoint: `${DOGE_TXID}:1`, address: DOGE_ADDRESS, valueAtomic: '480000000000', coinbase: false }, - { indexAtomic: '1', previousOutpoint: `${DOGE_TXID}:0`, address: 'DKuVPFPZUwMcNCU9hYzB4Mp3cRVJPuZLyq', valueAtomic: '120000000000', coinbase: false }, + { + indexAtomic: '0', previousOutpoint: `${DOGE_TXID}:1`, address: DOGE_ADDRESS, valueAtomic: '480000000000', coinbase: false, + // The authority keeps no inventory for a spent output, so an input + // is out of coverage rather than a fabricated empty inventory. + assets: { positions: [], coverage: 'out-of-coverage', coveredProtocolIds: [] }, + }, + { + indexAtomic: '1', previousOutpoint: `${DOGE_TXID}:0`, address: 'DKuVPFPZUwMcNCU9hYzB4Mp3cRVJPuZLyq', valueAtomic: '120000000000', coinbase: false, + assets: { positions: [], coverage: 'out-of-coverage', coveredProtocolIds: [] }, + }, ], outputs: [ - { indexAtomic: '0', address: 'DPMFn6Yv9GsuTPZWCbTFmZ9UWjjjjcYfWr', valueAtomic: '525000000000', spent: false }, - { indexAtomic: '1', address: DOGE_ADDRESS, valueAtomic: '74886600000', spent: true }, + { + indexAtomic: '0', address: 'DPMFn6Yv9GsuTPZWCbTFmZ9UWjjjjcYfWr', valueAtomic: '525000000000', spent: false, + // One unspent output carrying two protocols at once: a Doginal + // inscription and a dune balance with its own divisibility. + assets: { + positions: [ + { + outpoint: `${DOGE_TXID}:0`, vout: 0, valueSatsAtomic: '525000000000', + asset: { protocolId: 'doginals', assetId: `${DOGE_TXID}i0`, assetKind: 'inscription' }, + ownerAddress: 'DPMFn6Yv9GsuTPZWCbTFmZ9UWjjjjcYfWr', state: 'active', + evidence: { authorityId: 'ord-dogecoin', protocolId: 'doginals', releaseSha: 'ord-dogecoin-0.19', schemaVersion: 'ord-dogecoin-json-api', coverage: 'complete', negativeCompleteness: true, checkpoint: { chain: 'dogecoin', network: 'mainnet', heightAtomic: '5623041', blockHash: DOGE_BLOCK, reorgEpoch: '0', observedAt: '2026-08-29T04:05:00.000Z' }, checkedAt: '2026-08-29T04:05:00.000Z' }, + }, + { + outpoint: `${DOGE_TXID}:0`, vout: 0, valueSatsAtomic: '525000000000', + asset: { protocolId: 'dunes', assetId: 'VERY•MUCH•DUNE', displayName: 'VERY•MUCH•DUNE', ticker: '▪', assetKind: 'fungible', decimals: 8 }, + quantityAtomic: '250000000000', + ownerAddress: 'DPMFn6Yv9GsuTPZWCbTFmZ9UWjjjjcYfWr', state: 'active', + evidence: { authorityId: 'ord-dogecoin', protocolId: 'dunes', releaseSha: 'ord-dogecoin-0.19', schemaVersion: 'ord-dogecoin-json-api', coverage: 'complete', negativeCompleteness: true, checkpoint: { chain: 'dogecoin', network: 'mainnet', heightAtomic: '5623041', blockHash: DOGE_BLOCK, reorgEpoch: '0', observedAt: '2026-08-29T04:05:00.000Z' }, checkedAt: '2026-08-29T04:05:00.000Z' }, + }, + ], + coverage: 'complete', + coveredProtocolIds: ['doginals', 'dunes'], + }, + }, + { + indexAtomic: '1', address: DOGE_ADDRESS, valueAtomic: '74886600000', spent: true, + assets: { positions: [], coverage: 'out-of-coverage', coveredProtocolIds: [] }, + }, ], }, shielded: null, protocolActions: { candidates: [], confirmed: [ - { eventId: 'doginals:1', protocolId: 'doginals', state: 'confirmed-accepted', actionType: 'inscribe', evidenceIds: ['ord-dogecoin:5623038'] }, + { + eventId: 'doginals:1', protocolId: 'doginals', state: 'confirmed-accepted', actionType: 'inscribe', evidenceIds: ['ord-dogecoin:5623038'], + asset: { protocolId: 'doginals', assetId: `${DOGE_TXID}i0`, assetKind: 'inscription' }, + outputOutpoints: [`${DOGE_TXID}:0`], + }, { eventId: 'drc20:1', protocolId: 'drc20', state: 'confirmed-rejected', actionType: 'mintOverCap', evidenceIds: ['ord-dogecoin:5623038'] }, ], }, evidenceIds: ['dogecoin-blockbook:transaction'], completeness: 'complete', + assetFlow: { + checkpoint: { chain: 'dogecoin', network: 'mainnet', heightAtomic: '5623041', blockHash: DOGE_BLOCK, reorgEpoch: '0', observedAt: '2026-08-29T04:05:00.000Z' }, + complete: false, + unknownAttachmentCount: 0, + outOfCoverageCount: 3, + coveredProtocolIds: ['doginals', 'dunes'], + }, }; } @@ -385,6 +430,64 @@ export const chainFixtures = { ], }, + // The asset-holdings view the address page requests beside the base view: + // one output carrying a dune, one proven empty, a DRC-20 ledger balance at + // address level, and aggregates that never double count the two sections. + [`/api/v1/dogecoin/address/${DOGE_ADDRESS}/holdings`]: { + schemaVersion: 'universe-address-holdings-v1', + chain: 'dogecoin', + network: 'mainnet', + address: DOGE_ADDRESS, + utxos: [ + { + outpoint: `${DOGE_TXID}:1`, txid: DOGE_TXID, vout: 1, valueAtomic: '74886600000', + address: DOGE_ADDRESS, heightAtomic: '5623038', confirmationsAtomic: '3', + assets: { + positions: [ + { + outpoint: `${DOGE_TXID}:1`, vout: 1, valueSatsAtomic: '74886600000', + asset: { protocolId: 'dunes', assetId: 'VERY•MUCH•DUNE', displayName: 'VERY•MUCH•DUNE', ticker: '▪', assetKind: 'fungible', decimals: 8 }, + quantityAtomic: '90000000000', ownerAddress: DOGE_ADDRESS, state: 'active', + evidence: { authorityId: 'ord-dogecoin', protocolId: 'dunes', releaseSha: 'ord-dogecoin-0.19', schemaVersion: 'ord-dogecoin-json-api', coverage: 'complete', negativeCompleteness: true, checkpoint: { chain: 'dogecoin', network: 'mainnet', heightAtomic: '5623041', blockHash: DOGE_BLOCK, reorgEpoch: '0', observedAt: '2026-08-29T04:05:00.000Z' }, checkedAt: '2026-08-29T04:05:00.000Z' }, + }, + ], + coverage: 'complete', + coveredProtocolIds: ['doginals', 'dunes', 'drc20'], + }, + }, + { + outpoint: `${DOGE_BLOCK}:0`, txid: DOGE_BLOCK, vout: 0, valueAtomic: '1209573400000', + address: DOGE_ADDRESS, heightAtomic: '5620112', confirmationsAtomic: '2929', + assets: { positions: [], coverage: 'proven-empty', coveredProtocolIds: ['doginals', 'dunes', 'drc20'] }, + }, + ], + addressLevelBalances: [ + { + asset: { protocolId: 'drc20', assetId: 'dogi', ticker: 'dogi', assetKind: 'fungible', decimals: 18 }, + quantityAtomic: '2500000000000000000000', + availableAtomic: '1500000000000000000000', + transferableAtomic: '1000000000000000000000', + decimals: 18, + semantics: 'drc20-address-ledger', + evidence: { authorityId: 'ord-dogecoin', protocolId: 'drc20', releaseSha: 'ord-dogecoin-0.19', schemaVersion: 'ord-dogecoin-json-api', coverage: 'complete', negativeCompleteness: true, checkpoint: { chain: 'dogecoin', network: 'mainnet', heightAtomic: '5623041', blockHash: DOGE_BLOCK, reorgEpoch: '0', observedAt: '2026-08-29T04:05:00.000Z' }, checkedAt: '2026-08-29T04:05:00.000Z' }, + }, + ], + aggregateHoldings: [ + { + asset: { protocolId: 'dunes', assetId: 'VERY•MUCH•DUNE', displayName: 'VERY•MUCH•DUNE', ticker: '▪', assetKind: 'fungible', decimals: 8 }, + quantityAtomic: '90000000000', + utxoCountAtomic: '1', + source: 'utxo-bound', + }, + ], + paging: { limitAtomic: '100', offsetAtomic: '0', returnedAtomic: '2', totalUtxoCountAtomic: '2', hasMore: false }, + checkpoint: { chain: 'dogecoin', network: 'mainnet', heightAtomic: '5623041', blockHash: DOGE_BLOCK, reorgEpoch: '0', observedAt: '2026-08-29T04:05:00.000Z' }, + sourceEvidence: [], + complete: true, + unknownAttachmentCount: 0, + outOfCoverageCount: 0, + }, + // A real Zcash block, exactly as production returned it. The transaction // list is kept whole: trimming it left the page saying page 1 of 1 over // four of the seventeen ids the same response counts, which is a false @@ -488,6 +591,88 @@ export const chainFixtures = { "chain": "zcash" }, + // The Zcash address holdings view: a ZRune on one output with its own + // divisibility, a Zerdinal on another, one unscanned output that must not + // read as empty, and ZRC-20 ledger balances at address level. + [`/api/v1/zcash/address/${ZEC_ADDRESS}/holdings`]: { + schemaVersion: 'universe-address-holdings-v1', + chain: 'zcash', + network: 'mainnet', + address: ZEC_ADDRESS, + utxos: [ + { + outpoint: 'e642bab049488871a307aff5a46be789b374858dfa7163e33ddff27c76e80438:0', + txid: 'e642bab049488871a307aff5a46be789b374858dfa7163e33ddff27c76e80438', + vout: 0, valueAtomic: '125522000', address: ZEC_ADDRESS, + assets: { + positions: [ + { + outpoint: 'e642bab049488871a307aff5a46be789b374858dfa7163e33ddff27c76e80438:0', + vout: 0, valueSatsAtomic: '125522000', + asset: { protocolId: 'zrunes', assetId: '2107254:1', displayName: 'ZCASH•FOREVER', ticker: 'Ⓩ', assetKind: 'fungible', decimals: 2 }, + quantityAtomic: '1500000', ownerAddress: ZEC_ADDRESS, state: 'active', + evidence: { authorityId: 'index-zcash-metaprotocols', protocolId: 'zrunes', releaseSha: 'index-zcash-metaprotocols', schemaVersion: 'zcash-metaprotocols-api-v1', coverage: 'complete', negativeCompleteness: true, checkpoint: { chain: 'zcash', network: 'mainnet', heightAtomic: '3464734', blockHash: '0000000000645e18fd55f18a5b2681f0becdc180d14d35a730875c4dc66add1a', reorgEpoch: '0', observedAt: '2026-08-30T00:00:00.000Z' }, checkedAt: '2026-08-30T00:00:00.000Z' }, + }, + ], + coverage: 'complete', + coveredProtocolIds: ['zerdinals', 'zrunes', 'zrc721'], + }, + }, + { + outpoint: '44b60d006a4197f81385da1718eeddb5be23fdfdaa45553f0cae97766f2e150e:0', + txid: '44b60d006a4197f81385da1718eeddb5be23fdfdaa45553f0cae97766f2e150e', + vout: 0, valueAtomic: '125015000', address: ZEC_ADDRESS, + assets: { + positions: [ + { + outpoint: '44b60d006a4197f81385da1718eeddb5be23fdfdaa45553f0cae97766f2e150e:0', + vout: 0, valueSatsAtomic: '125015000', + asset: { protocolId: 'zerdinals', assetId: '44b60d006a4197f81385da1718eeddb5be23fdfdaa45553f0cae97766f2e150ei0', assetKind: 'inscription' }, + ownerAddress: ZEC_ADDRESS, state: 'active', + evidence: { authorityId: 'index-zcash-metaprotocols', protocolId: 'zerdinals', releaseSha: 'index-zcash-metaprotocols', schemaVersion: 'zcash-metaprotocols-api-v1', coverage: 'complete', negativeCompleteness: true, checkpoint: { chain: 'zcash', network: 'mainnet', heightAtomic: '3464734', blockHash: '0000000000645e18fd55f18a5b2681f0becdc180d14d35a730875c4dc66add1a', reorgEpoch: '0', observedAt: '2026-08-30T00:00:00.000Z' }, checkedAt: '2026-08-30T00:00:00.000Z' }, + }, + ], + coverage: 'complete', + coveredProtocolIds: ['zerdinals', 'zrunes', 'zrc721'], + }, + }, + { + outpoint: '69fb0df1609d51da8f8526bf28fd7d92213461609811e3e119cc88623e93e5a5:0', + txid: '69fb0df1609d51da8f8526bf28fd7d92213461609811e3e119cc88623e93e5a5', + vout: 0, valueAtomic: '125176000', address: ZEC_ADDRESS, + // An output the scanner has not reached: unknown, never empty. + assets: { positions: [], coverage: 'unscanned', coveredProtocolIds: ['zerdinals', 'zrunes', 'zrc721'] }, + }, + ], + addressLevelBalances: [ + { + asset: { protocolId: 'zrc20', assetId: 'zord:zero', ticker: 'ZERO', assetKind: 'fungible', decimals: 8 }, + quantityAtomic: '4200000000', + availableAtomic: '4200000000', + transferableAtomic: '0', + decimals: 8, + semantics: 'zrc20-zord-ledger', + evidence: { authorityId: 'index-zcash-metaprotocols', protocolId: 'zrc20', releaseSha: 'index-zcash-metaprotocols', schemaVersion: 'zcash-metaprotocols-api-v1', coverage: 'complete', negativeCompleteness: true, checkpoint: { chain: 'zcash', network: 'mainnet', heightAtomic: '3464734', blockHash: '0000000000645e18fd55f18a5b2681f0becdc180d14d35a730875c4dc66add1a', reorgEpoch: '0', observedAt: '2026-08-30T00:00:00.000Z' }, checkedAt: '2026-08-30T00:00:00.000Z' }, + }, + ], + aggregateHoldings: [ + { + asset: { protocolId: 'zerdinals', assetId: '44b60d006a4197f81385da1718eeddb5be23fdfdaa45553f0cae97766f2e150ei0', assetKind: 'inscription' }, + quantityAtomic: '1', utxoCountAtomic: '1', source: 'utxo-bound', + }, + { + asset: { protocolId: 'zrunes', assetId: '2107254:1', displayName: 'ZCASH•FOREVER', ticker: 'Ⓩ', assetKind: 'fungible', decimals: 2 }, + quantityAtomic: '1500000', utxoCountAtomic: '1', source: 'utxo-bound', + }, + ], + paging: { limitAtomic: '100', offsetAtomic: '0', returnedAtomic: '3', totalUtxoCountAtomic: '3', hasMore: false }, + checkpoint: { chain: 'zcash', network: 'mainnet', heightAtomic: '3464734', blockHash: '0000000000645e18fd55f18a5b2681f0becdc180d14d35a730875c4dc66add1a', reorgEpoch: '0', observedAt: '2026-08-30T00:00:00.000Z' }, + sourceEvidence: [], + complete: false, + unknownAttachmentCount: 1, + outOfCoverageCount: 0, + }, + // A Zcash address, as production returned it, with the unspent output list // and the history trimmed. The balance is nested and in snake_case, the // history counts from an offset, and an unspent output states its index as a