Refactor hwygen arm64 asm dispatch ownership - #65
Merged
Conversation
Replace the roaring-only regenerate + git-diff-exit-code check with a full `go generate ./hwy/contrib/...` on ARM64. The bit-identical diff check was failing because CI's clang produces different register allocation than the macOS clang used to check in the assembly. Now CI regenerates and tests all contrib packages (matmul, vec, nn, etc.) on ARM64, catching hwygen regressions without requiring deterministic assembly output.
GoAT compiles C → .s → .o, then runs objdump on the .o to get binary encodings. When cross-compiling to a different OS (e.g. darwin from linux), the .o was in Mach-O format which the Linux system objdump can't read. Fix: when the target OS differs from the host OS (but same arch), assemble the .o using the host OS target triple. The instructions are identical — only the container format (ELF vs Mach-O) changes — so objdump can read it and extract the same binary encodings. Also regenerate all contrib packages on ARM64 CI, not just roaring.
ajroetker
force-pushed
the
hwygen-arm64-dispatch-cleanup
branch
from
March 23, 2026 05:46
1bfa628 to
59ae7d1
Compare
When cross-compiling to a different OS (e.g. darwin from linux), the .s file uses target-specific assembly syntax (Mach-O directives, ; comments) that the host assembler can't parse. Instead, compile the .o directly from the C source using the host OS triple. The instructions are identical regardless of OS, so objdump extracts the same binary encodings. Also revert the llvm-objdump approach since it has incompatible output format (different symbol names, leading whitespace).
When cross-compiling (e.g. generating Darwin SME assembly on Linux), the system objdump can't read Mach-O object files. GoAT now falls back to llvm-objdump, which handles both Mach-O and ELF formats. llvm-objdump emits synthetic symbol names (e.g. "ltmp0") for the first function in Mach-O files. The ARM64 objdump parser now accepts an ordered function name list and falls back to positional matching when a symbol isn't found in the function map. Install llvm on CI runners to provide llvm-objdump.
Different clang versions produce functionally different assembly (different optimization choices, register allocation), which causes test failures in floating-point precision tests. The checked-in assembly is already tested in the earlier CI steps. The regeneration step now just verifies the hwygen + GoAT pipeline produces compilable code.
When GoAT fails to compile a C file (e.g., bf16 NEON ops unsupported by the host clang), it may have already generated .go and .s stubs before clang failed. These orphaned files conflict with checked-in .gen.go files, causing "redeclared in this block" errors. Remove all intermediate files (.c, .o, .go, .s) in the error cleanup path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Testing