Skip to content

Retry transient Zig ARM64 release allocation failures - #452

Merged
ajroetker merged 1 commit into
mainfrom
codex/retry-zig-arm64-release
Aug 6, 2026
Merged

Retry transient Zig ARM64 release allocation failures#452
ajroetker merged 1 commit into
mainfrom
codex/retry-zig-arm64-release

Conversation

@ajroetker

Copy link
Copy Markdown
Contributor

Summary

  • wrap Zig release build steps so their first-attempt output can be classified
  • retry once only for aarch64-linux-musl + ReleaseSmall allocation failures
  • preserve the populated local cache for the retry
  • fail immediately for every unrelated compiler or source error

Why

The ARM64 release compile repeatedly aborts in Zig/LLVM allocation handling despite zero kernel/cgroup OOM events. A 64 GB runner failed too. In the isolated diagnostic, the fresh release compile failed with LLVM ERROR: out of memory / Buffer allocation failed, then both the full build-runner replay and direct compiler replay succeeded from independent copies of the partially populated local cache at essentially the same ~15.9 GB max RSS and zero swap.

Diagnostic evidence: #450 and cache A/B run.

This is a bounded release mitigation, not a claim that the upstream Zig/LLVM issue is fixed.

Verification

  • bash -n scripts/packaging/build_zig_release_archive.sh
  • git diff --check
  • fake-compiler wrapper test: recognized allocation failure called install twice, then lite-capi, and produced a valid archive
  • fake-compiler wrapper test: unrelated compilation failure called install once and exited 1 without retrying

@ajroetker

Copy link
Copy Markdown
Contributor Author

End-to-end ARM64 validation passed on the original normal runner: run 31059377379.

Observed real wrapper behavior:

  1. First install failed with std::bad_alloc.
  2. The new classifier matched it and emitted the bounded retry.
  3. Second install succeeded using the populated local cache.
  4. lite-capi succeeded without a retry.
  5. Staging, archive content validation, and artifact upload all succeeded.

The runner remained the original 32 GB machine / 24,000 Mi request. PR checks are green and the PR is mergeable.

@ajroetker
ajroetker merged commit 3039ac6 into main Aug 6, 2026
9 checks passed
@ajroetker
ajroetker deleted the codex/retry-zig-arm64-release branch August 6, 2026 00:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant