Skip to content

fix(setup): drain child pipes while the child runs, not after it exits - #529

Merged
thcp merged 1 commit into
0.16.1from
fix/516-drain-child-pipes
Aug 31, 2026
Merged

fix(setup): drain child pipes while the child runs, not after it exits#529
thcp merged 1 commit into
0.16.1from
fix/516-drain-child-pipes

Conversation

@thcp

@thcp thcp commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

Fixes #516. Targets fix/510-updater-url-allowlist -- both touch main.rs and the plan sequences the Rust changes. Merge #528 first.

The problem

child_output_with_timeout read the pipes only after try_wait() reported an exit:

loop {
    if let Some(status) = child.try_wait()? {
        // read_to_end on stdout/stderr -- only reachable once the child is gone
    }
    thread::sleep(Duration::from_millis(100));
}

A child that outruns the OS pipe buffer blocks in write() with nobody reading, so it never exits, so try_wait() never reports an exit. The call ends at the timeout with no output.

warmup_models pipes both streams (main.rs:1414-1415) and its model downloads emit tqdm progress to stderr in proportion to how long they take, not how large they are. The failure therefore lands on slow connections -- exactly the users warmup exists to spare a mid-pipeline download.

Change

Each stream drains on its own thread. Both are required: draining one then the other reintroduces the deadlock on whichever is second. Readers are joined rather than detached on the timeout path, since killing the child closes its ends and dropping the handles would leak two threads per timeout.

command_output_with_timeout had the identical shape and now delegates, so the two cannot drift apart again.

Verification -- the deadlock is reproduced, not assumed

New test a_chatty_child_is_drained_rather_than_deadlocked writes 512 KiB to each stream.

implementation result
read-after-exit (old) FAILED -- "chatty child timed out after 20 seconds", took 20.09s
concurrent draining (this PR) passed in 0.11s

Worth recording: when I filed #516 I measured a real cold-cache warmup on a fast connection and found only 8,360 bytes of stderr -- under the buffer. That is why this had not been hit in practice, not why it could not happen. The test settles it.

cargo fmt --check   OK
cargo clippy        0 errors
cargo test          59 passed, 1 failed

The 1 failure is tests::a_free_port_is_granted_as_asked, the known parallel-execution flake, failing 3/3 on the base branch without this change.

One thing to check in review

While rewriting command_output_with_timeout I initially sliced out the #[cfg(windows)] attribute above hide_console_window, which cargo check caught immediately. Restored, and both hide_console_window definitions keep their cfg guards -- worth a glance since a Linux/macOS-only compile would not exercise the Windows one.

@thcp
thcp force-pushed the fix/510-updater-url-allowlist branch from 6a05da5 to a404c7f Compare August 31, 2026 21:05
Base automatically changed from fix/510-updater-url-allowlist to 0.16.1 August 31, 2026 21:09
child_output_with_timeout read stdout and stderr only once try_wait() had
reported an exit. A child that outruns the OS pipe buffer blocks in write()
with nobody reading, so it never exits, so try_wait() never reports an exit,
and the call ends at the timeout with no output at all.

warmup_models pipes both streams, and its model downloads emit tqdm progress
to stderr in proportion to how long they take rather than how large they are.
So the failure lands on slow connections -- the users warmup exists to spare a
mid-pipeline download.

Each stream now drains on its own thread. Both are needed: draining one and
then the other reintroduces the deadlock on whichever is second. The readers
are joined rather than detached on the timeout path too, since killing the
child closes its ends and dropping the handles would leak two threads per
timeout.

command_output_with_timeout had the same shape and now delegates, so the two
cannot drift apart again.

The test writes 512 KiB to each stream, comfortably past any pipe buffer.
Against the old implementation it fails after burning the full 20-second
timeout; with concurrent draining it completes in 0.11s. Worth noting because
a measurement of a real cold-cache warmup on a fast connection showed only
8,360 bytes of stderr -- under the buffer, which is why this had not been seen
in practice rather than why it could not happen.

Refs #516
@thcp
thcp force-pushed the fix/516-drain-child-pipes branch from e24e0a7 to 845b5ac Compare August 31, 2026 21:10
@thcp
thcp marked this pull request as ready for review August 31, 2026 21:13
@thcp
thcp merged commit ebbdb44 into 0.16.1 Aug 31, 2026
10 checks passed
@thcp
thcp deleted the fix/516-drain-child-pipes branch August 31, 2026 21:13
thcp added a commit that referenced this pull request Aug 31, 2026
Fixes #518. **Targets `fix/516-drain-child-pipes`** -- last of the
sequential Rust changes. Merge #529 first.

## The problem

`download_linux_ffmpeg` fetched a tarball from a rolling URL on a single
non-CDN host, extracted it with the system `tar`, marked the binaries
executable and ran them -- with **no integrity check of any kind**.

- **Windows** verifies against BtbN's published `checksums.sha256`
- **macOS** verifies against four pinned hashes
- **Linux** verified nothing

`verify_ffmpeg` only proves the binary runs and has the encoders
StemDeck needs. That says nothing about where it came from.

## Why a pinned SHA256 rather than upstream's .md5

The obvious move was the `.md5` companion, matching the Windows shape. I
checked it and it is worth less than it looks:

- **MD5 is broken for collisions**
- **The companion is served by the same host as the tarball.** Anyone
able to replace one can replace the other. It evidences corruption, not
authenticity.
- Upstream publishes no `.sha256` (confirmed: 404)

So this pins our own SHA256, the way the macOS path already does. The
pinned value was computed from the artifact whose MD5 matched upstream's
published `7fa72b652e19bf84c9461e332ea1cdf3`, so the pin is anchored to
what upstream currently vouches for.

```
url    .../ffmpeg-release-amd64-static.tar.xz   (last-modified 2024-08-24)
md5    7fa72b652e19bf84c9461e332ea1cdf3          (matches upstream)
sha256 abda8d77ce8309141f83ab8edf0596834087c52467f6badf376a6a2a4c87cf67
```

**Trade-off:** the URL is rolling, so this needs a manual bump when
upstream publishes a new build. A stale pin fails closed with a checksum
error rather than silently accepting whatever arrives, which is the
right direction to fail. `STEMDECK_FFMPEG_URL` still overrides and skips
the check -- an override points somewhere we cannot have a hash for,
matching how the macOS override already behaves.

## Verification -- and a gap reviewers should know about

**This code cannot be compiled on macOS**, and **`ci.yml` does not build
the Rust shell at all** -- Linux Rust is compiled only by
`linux-release.yml`, at release time. So a type error here would first
surface during a release.

To check it locally I temporarily widened the cfg gate on the two
constants and `download_linux_ffmpeg` from `#[cfg(all(unix,
not(target_os = "macos")))]` to `#[cfg(unix)]` and ran `cargo check`:

```
error count: 0
```

Then restored all 7 gates (verified by count).

```
cargo fmt --check   OK
cargo clippy        0 errors
cargo test          59 passed, 1 failed (the known port flake, fails on base too)
```

## Worth filing separately

There is no `linux-check.yml`. #421 added `macos-check.yml` and
`windows-check.yml` precisely because `ci.yml` is 100% ubuntu and could
not compile cfg-gated code for those platforms -- but nothing compiles
the **Linux** Rust either, despite the runner being ubuntu. That is a
real hole and it is what made this PR awkward to verify. Happy to open
an issue.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant