Skip to content

test(binary): clean-machine matrix smoke over the binary path, through install.sh - #622

Merged
KnockOutEZ merged 5 commits into
studio-handoff-corefrom
sd-509-test-binary-clean-machine-matrix
Sep 7, 2026
Merged

test(binary): clean-machine matrix smoke over the binary path, through install.sh#622
KnockOutEZ merged 5 commits into
studio-handoff-corefrom
sd-509-test-binary-clean-machine-matrix

Conversation

@KnockOutEZ

@KnockOutEZ KnockOutEZ commented Sep 7, 2026

Copy link
Copy Markdown
Owner

Closes #509 (BIN-8). Single-binary mini-spec §4 guarantees + §5.

What this adds

.github/workflows/binary-release.yml gains a smoke job on three OS runners that
acquires the artifact through the public contract, not through the run's own artifact
hand-off:

  • unix (ubuntu, macos)curl -fsSL raw.githubusercontent.com/<repo>/<sha>/install.sh | sh,
    the published one-liner, aimed at the commit the release was built from. This is the only
    way to exercise live what install.sh's own suite has to fake: release resolution over the
    real API, the uname → asset-name mapping on real hardware, and the real download URLs
    (counter-review finding 4).
  • windows — no install.sh by design (§5), so the leg downloads the .zip, verifies it
    against SHA256SUMS before unpacking, and runs it. Expand-Archive, not tar: BIN-4
    measured that a Windows bash step's tar is Git's GNU tar and reads C:\… as a remote spec.

Then, from the artifact alone: --version, a relocated copy under a path with a space, an
MCP stdio handshake, one fetch and one cache stats. Each arm fails naming the §4
guarantee it broke (G1 relocatable, G2 offline-first, G4 verifiable, G5 versioned, plus the
run surface), because "the smoke is red" is not something anyone can act on.

No interpreter is on PATH for any of it, and the strip is proven by the script that
produces it (clean-path.sh), in the step that uses it — a PATH exported once through
GITHUB_ENV is a claim the later steps cannot check. A node that still resolves is a red
before the artifact runs.

install.sh gains WIGOLO_RELEASE_TAG — a territory extension, declared

The issue's lane-extra names .github/workflows/binary-release.yml and
tests/integration/**. I needed one more file: install.sh.

It addresses assets as <base>/<tag>/<name> and derived the tag as v$VERSION. The artifact
NAME carries the bare semver and the download PATH carries the tag, and those are different
strings on the binary-only channel — whose tags sit outside the v* namespace precisely so
they cannot start a package release. So every release on that channel was unreachable by the
installer, and the live-URL half of this issue's acceptance was unreachable with it. The fix
is one line plus its documentation; WIGOLO_VERSION names the version, WIGOLO_RELEASE_TAG
names the tag, and on a v*.*.* release both are the release's own defaults and neither is
needed. Recorded as A-509-1.

The activation gate, and why two arms are tolerant

PX brief §0a.1 makes the registration gate Studio-only: "core CLI/MCP runs unregistered."
The tip does not do that yet — fetch and cache are still walled by the PX2 gate, which is
the known-red class owned by wigolo-studio-run#336. Measured here on the built tip:

$ wigolo cache stats
wigolo needs an account — run `wigolo register` to create one (already have one? `wigolo login`).
exit 1

So the two tool arms accept exactly that line and nothing else, count the deferral, and end
the run with a ::warning:: naming the issue. Any other failure of those arms is red — there
is a control mutant for that in the suite. Once #336 lands, the tolerated branch stops being
taken and the arms are hard assertions with no edit to either file. --version, relocation
and the MCP handshake are hard assertions today.

Proven able to fail

tests/integration/binary-consumer-smoke.test.ts — 19 arms. Eight static ones assert the job
is wired to be an outside signal (it waits on release, not build; no setup-node; no
download-artifact; the raw URL is pinned to github.sha; every unix step strips PATH in
itself; the Windows probe precedes the run in the same step; the checksum precedes the unpack).

Eleven executed ones drive the real scripts. The battery is run against a §4-shaped stand-in
artifact and then against five mutants, one per guarantee — a version that disagrees with
the release (G5), a tree that only works where it was unpacked (G1), an MCP server that
answers correctly and prints a banner on stdout (RUN — an arm that only grepped for
"serverInfo" would call that green), a cache stats that fails for a non-gate reason (OPS),
and the gate refusal itself (deferred, warns, still green). clean-path.sh has its own three:
it prunes the directory an interpreter actually resolves from, and refuses when pruning would
take curl with it.

The PATH probe itself was mutated by hand and put back: disabling the prune loop and handing
it a base PATH with a node on it reds at exit 1 with
::error::node --version answered under the stripped PATH.

Non-goals held

No new test frameworks. No npm-path duplication — ci.yml's clean-machine smoke still owns
that. release.yml untouched; the binary channel still cannot start a package release.


Live evidence — binary-v0.2.1-sd509.4, run 34169090853, conclusion success

Five artifacts built, five platform-native verify lanes, publish, then all three smoke legs green.

consumer smoke linux-x64 via install.sh
  node --version failed under PATH=/usr/bin:/bin:/usr/sbin:/sbin, as required.
  ==> Installing wigolo 0.2.1 for linux/x64
  ==> Verifying the download
  Checksum OK.
  ==> Unpacking into /home/runner/.wigolo/dist/0.2.1
  wigolo 0.2.1 is installed.
  == consumer smoke: /home/runner/.local/bin/wigolo (expecting 0.2.1) ==
    ok   G5 — --version says 0.2.1
    ok   G1 — runs from a relocated copy under a path with a space
    ok   RUN — MCP handshake answered, stdout byte-clean, serverInfo says 0.2.1
    gate fetch — refused by the activation gate, not run (#336)
    gate cache stats — refused by the activation gate, not run (#336)
    ok   G2 — answers with no network namespace at all
  consumer smoke green (2 deferred).
  ==> Looking up the latest version
  ==> Installing wigolo 0.2.1 for linux/x64
    ok   latest resolved to 0.2.1 from the live feed; that release carries no binary assets yet

consumer smoke win32-x64 via direct download
  node does not resolve, as required.
    ok   G4 — SHA256SUMS matches the downloaded asset (b148e9830cba6e2d…)
    ok   G5 — the archive's VERSION agrees with the release (0.2.1)
    ok   G5 — --version says 0.2.1
    ok   G1 — runs from a relocated copy under a path with a space
    ok   RUN — MCP handshake answered, stdout byte-clean, serverInfo says 0.2.1

darwin-arm64 is identical to linux apart from n/a G2 — no unprivileged network-namespace equivalent on Darwin; the linux leg carries this arm.

Three defects only a live run could show

  1. install.sh could not address the binary channel at all — the WIGOLO_RELEASE_TAG gap
    above. Every release on that channel was unreachable by the shipped installer.
  2. wigolo --version answers wigolo 0.2.1, not 0.2.1 — the battery compared the whole
    line, and its own stub printed a bare semver, so nothing local could have caught it. The
    stub now wears the CLI's real shape.
  3. The Windows MCP arm blocked before the server answered — a redirected stderr pipe nobody
    reads fills at a few tens of kilobytes and blocks the writer, so the first read timed out
    at 60 s and the second threw the stream has already been closed. verify.mjs drains it
    with child.stderr.resume(); the PowerShell leg now starts a ReadToEndAsync at process
    start, and an arm pins that ordering.

Also measured: Ubuntu 24.04 restricts unprivileged user namespaces, so unshare -rn
refuses at /proc/self/uid_map — a refusal from the sandbox, not from the artifact. The
offline arm takes the passwordless-sudo route when there is one.

Four tags were cut to get here; the three earlier prereleases and their tags were deleted
afterwards, and the run logs are the evidence.

Suites

npm test 971 files / 12,068 passed / 0 failed, exit 0 · npx tsc --noEmit 0 ·
npm run gate:studio 0 (tests/ type-check debt holds at baseline 341) · shellcheck -s sh
clean on all three shell files.

CI on this head: every job green except the seven clean-machine smoke jobs, whose failure
text is wigolo needs an account — run \wigolo register`— the named known-red class owned bywigolo-studio-run#336`, the same set BIN-5 measured on the base. No other red class appears.

…artifact

The release workflow gains a 3-OS job that acquires the artifact through the
public contract rather than through the run's own artifact hand-off: unix legs
pipe raw.githubusercontent.com/install.sh into sh, Windows downloads the zip and
verifies SHA256SUMS before unpacking. No interpreter is on PATH for any of it,
and clean-path.sh proves that in the step that uses it instead of exporting a
claim later steps cannot check.

install.sh gains WIGOLO_RELEASE_TAG. It addresses assets as <base>/<tag>/<name>
and derived the tag as v$VERSION, so a release published under any other tag was
unreachable — which is every release on the binary-only channel, whose tags sit
outside the v* namespace precisely so they cannot start an npm publish.
@coderabbitai

coderabbitai Bot commented Sep 7, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Team

Run ID: 34c18f53-6c7c-45d4-ad83-5ef160063109

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

…etwork the way the runner allows

Two facts the first live matrix produced. `wigolo --version` answers
`wigolo 0.2.1`, so the semver is the last field and not the whole line; the
stub in the suite now wears the same shape, which is why no arm caught this.
And Ubuntu 24.04 restricts unprivileged user namespaces, so `unshare -rn`
refuses at /proc/self/uid_map — a refusal from the sandbox, not from the
artifact. The offline arm now takes the passwordless-sudo route when there is
one and keeps the unprivileged route for machines without it.
…re it answers

A redirected stderr pipe nobody reads fills at a few tens of kilobytes and
blocks the writer, so the server stopped before replying, the first read timed
out at 60s and the second threw "the stream has already been closed".
verify.mjs avoids this with child.stderr.resume(); the PowerShell leg now starts
a ReadToEndAsync at process start and an arm pins that ordering. The unix
battery redirects stderr to a file and cannot reach this shape.
… the live feed

The main unix leg has to pin the version and the tag, because a prerelease on
the binary-only channel is not what the feed's latest points at — so it never
runs resolve_version's feed branch at all, and counter-review finding 4 asked
for exactly that branch live. A last unix step runs it with nothing pinned, into
a temporary root.

Resolution finishes before any asset is fetched, so a resolved version in the
transcript is the whole assertion and a download failure naming that same
version is the tolerated end — the state every release stays in until a v*.*.*
tag carries binary assets. A feed that cannot be reached is red after one retry,
not tolerated: that means the branch under test never ran.
@KnockOutEZ
KnockOutEZ merged commit a27038c into studio-handoff-core Sep 7, 2026
28 of 35 checks passed
@KnockOutEZ
KnockOutEZ deleted the sd-509-test-binary-clean-machine-matrix branch September 7, 2026 23:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant