Skip to content

Finish the container header on every rendered export - #473

Merged
thcp merged 2 commits into
mainfrom
fix/export-header-sizes
Aug 26, 2026
Merged

Finish the container header on every rendered export#473
thcp merged 2 commits into
mainfrom
fix/export-header-sizes

Conversation

@thcp

@thcp thcp commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator

Closes #458

Reported and diagnosed by @sam1am, whose commit is the first one here, pulled from his fork because the repo restricts pull requests to collaborators.

His analysis was exact. A muxer that finishes the file by seeking back to a header it wrote earlier cannot do that on pipe:1, so every WAV we exported kept the 0xFFFFFFFF placeholder and claimed roughly 4 GB of audio. The fix is his: render to a real file instead.

It reaches more than WAV

Measured through the real API, exporting the same four second mix in each format and running ffprobe on the result:

before after
wav Ignoring maximum wav data size, file may be invalid 4.00s, no warnings
flac Duration: N/A — STREAMINFO total samples and all 16 MD5 bytes zero 4.00s, no warnings
mp3 Duration: 00:00:03.87 — wrong, and Estimating duration from bitrate 4.00s, no warnings
ogg already correct unchanged, still streamed

MP3 is the one worth arguing about, so here is the argument. On a pipe ffmpeg does not write a wrong Xing frame, it writes none, and these are VBR (-q:a 2), so there is no duration and no seek table. The 3.87 above is what a player actually reports for a 4.00 second file.

get_stem_mp3, the MP3 region endpoint sitting directly beside the WAV one that was corrected, was still on pipe:1 and is now rendered too.

OGG stays streaming on purpose: a granule position rides on every page and nothing is patched afterwards. MP4 already muxes frag_keyframe+empty_moov for exactly this reason and is untouched.

Answering with a file, not a generator

_render_ffmpeg became _render_to_file and returns a path, so the endpoints reply with FileResponse.

The reason is not tidiness. A streamed render commits HTTP 200 before ffmpeg has exited, so a failure reached the client as a truncated file and only the server log knew. Awaiting the render first means the exit code is known while the response is still ours to choose, and a failed export is now an honest 500. That is the limitation #280 documented and could not fix from where it stood.

Content-Length and range requests come along with it, which chunked encoding cannot offer at all.

The cache would have kept serving the broken files

_mixdown_cache_key had no version component. Every entry written before this holds an unpatched header, the key that produced it is still reachable, and _prune_mixdown_cache evicts by age rather than validity. Without something, a user who exported before upgrading would be handed the same broken file back for the same parameters indefinitely.

_RENDER_CACHE_VERSION is folded into the key, so those entries can never be reached again and age out through the existing budget.

Verified

  • Python: 836 passed, against 825 on main run in a clean worktree with the same environment. The same four pre-existing local failures on both, none of them in stems or export.
  • Browser: 46 Playwright tests pass.
  • Lint: ruff check and ruff format --check clean.
  • End to end: exports pulled through the real API and run through ffprobe, which is the table above.

Each new test was confirmed to fail without the change it covers, not merely to pass with it. Putting MP3 back on the streaming path breaks the Xing assertions and the Content-Length one; restoring get_stem_mp3 to pipe:1 breaks the region test on its own.

One note for whoever runs these locally: 23 of the tests in test_stems_api.py skip silently without a resolvable ffmpeg, and they are the ones that check headers. STEMDECK_FFMPEG_DIR has to be set or the whole area looks green while testing nothing.

sam1am and others added 2 commits August 26, 2026 20:59
Builds on Sam's fix. His diagnosis was exact and the approach is his: a
container whose muxer finishes the file by seeking back cannot be written
to a pipe, so render it to a real file instead. Three things on top.

MP3 is affected as well. On a pipe ffmpeg does not write a wrong Xing
frame, it writes none at all, and these are VBR (-q:a 2), so the file
carries no duration and no seek table. Measured through the real API:
a four second export reported 00:00:03.87 before this and 00:00:04.00
after. get_stem_mp3, the MP3 region endpoint sitting next to the WAV one
that was corrected, was still on pipe:1 and is now rendered too. FLAC
was equally affected and equally unreported: STREAMINFO total samples
and all sixteen MD5 bytes were zero, and ffprobe read the duration as
N/A. ogg is deliberately left streaming, because a granule position
rides on every page and nothing is patched afterwards, and mp4 already
muxes fragmented for this exact reason.

_render_ffmpeg became _render_to_file, returning a path rather than
yielding chunks, so the endpoints answer with FileResponse. That is
worth more than the buffering costs. A streamed render commits HTTP 200
before ffmpeg has exited, so a failure reached the client as a truncated
file that only the log recorded; the exit code is now known while the
response is still ours to choose, and a failed render is an honest 500.
That is the limitation #280 documented and could not fix. Content-Length
and range requests come along with it.

The render cache had to be invalidated. Its key had no version, every
entry written before this holds an unpatched header, and
_prune_mixdown_cache evicts by age rather than validity, so anyone who
exported before upgrading would have been handed the same broken file
back for the same parameters indefinitely. _RENDER_CACHE_VERSION is
folded into the key so those entries can never be reached again.

Verified: 836 passed against 825 on main in a clean worktree, the same
four pre-existing local failures on both. 46 browser tests. Each new
test was confirmed to fail without the change it covers, by putting MP3
back on the streaming path and watching the Xing assertions break.
@thcp
thcp merged commit ab95be7 into main Aug 26, 2026
10 checks passed
@thcp
thcp deleted the fix/export-header-sizes branch August 26, 2026 20:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: RIFF and data chunk sizes incorrect on wav export

2 participants