Skip to content

ci: catch cross-platform build breaks before release#19

Open
chethanuk wants to merge 1 commit into
mainfrom
ci/cross-compile-guard
Open

ci: catch cross-platform build breaks before release#19
chethanuk wants to merge 1 commit into
mainfrom
ci/cross-compile-guard

Conversation

@chethanuk

Copy link
Copy Markdown
Owner

Description

CI builds one platform. ci.yml:52 is go build -o /dev/null ./cmd/opencodereview inside
container: golang:1.26.5, i.e. linux/amd64. release.yml:15-29 cross-compiles six, and
only fires on push: tags: ['v*'].

That gap has teeth here, because the repo carries platform-gated source CI never compiles:

cmd/opencodereview/shell_windows.go:1     //go:build windows
cmd/opencodereview/procattr_windows.go:1  //go:build windows
cmd/opencodereview/shell_unix.go:1        //go:build !windows
cmd/opencodereview/procattr_unix.go:1     //go:build !windows

procattr_unix.go uses syscall.SysProcAttr{Setpgid: true} and syscall.Kill; the Windows
twins are, as far as CI is concerned, dead code. A signature change to shellCommand or
configureProcessGroup compiles green on main and breaks at tag time — after the tag is
already public
.

All six targets build clean today. This is a missing guard, not a live bug.

I proved the gap rather than asserting it

Appended var _ = thisSymbolDoesNotExist to shell_windows.go and ran every gate CI has today
against every gate this PR adds:

Gate Exit Verdict
go build -o /dev/null ./cmd/opencodereview — today's Build step 0 green
go build -o /dev/null ./... — linux/amd64, whole tree 0 green
go vet ./... 0 green
GOOS=windows GOARCH=amd64 go build ./... — new matrix leg 1 cmd/opencodereview/shell_windows.go:14:9: undefined: thisSymbolDoesNotExist
GOOS=windows GOARCH=arm64 go build ./... — new matrix leg 1 same
GOOS=darwin GOARCH=arm64 go build ./... — new matrix leg 0 correctly unaffected

The first three rows are the argument: a break that ships broken binaries is invisible to
every check this repo runs today
. The last row shows fail-fast: false isolating the failure
to the targets that actually broke.

Cost — answered by production, not by my estimate

The obvious objection is runner time, and I'd rather not answer it with a number from my laptop.
release.yml:11-29 already runs this exact shape on this exact runner pool: runs-on: self-hosted, container: golang:1.26.5, a 6-way matrix over the same GOOS/GOARCH pairs,
CGO_ENABLED: '0'.

Measured from the last three releases (gh run view <id> --json jobs):

leg 29804216482 29721605700 29553313324
build (linux, amd64) 0.56 0.55 0.58 min
build (linux, arm64) 0.55 0.53 0.56 min
build (darwin, amd64) 0.58 0.60 0.63 min
build (darwin, arm64) 0.56 0.58 0.56 min
build (windows, amd64) 0.60 0.58 0.61 min
build (windows, arm64) 0.58 0.55 0.56 min

33–38 seconds per leg, including checkout, container start and artifact upload — and the legs
run in parallel. So the capacity question is already answered by your own release pipeline.

Two honest deltas: this job builds ./... where release.yml builds ./cmd/opencodereview
with -ldflags (locally that is ~+8%), and it skips artifact upload. Neither changes the order
of magnitude.

Why a separate job rather than a step in test

Appending five sequential cross-builds to test would put them inside its timeout-minutes: 15
budget, alongside go vet, a govulncheck install and scan, and go test -race over 23
packages. A hygiene PR that makes CI time out is the worst possible outcome. As a sibling job
the legs run concurrently, test's runtime is unchanged, and fail-fast: false names the
target that broke instead of surfacing one opaque loop failure.

linux/amd64 is deliberately excluded — test already covers it. To be sure that leaves no
hole, I checked that the linux/amd64 whole tree is covered too: injecting a break into a
non-cmd package (internal/llm/providers.go) makes go vet ./... exit 1. So vet and
test -race already compile all 23 packages on linux/amd64; the matrix only needs the other five.

Two riders, same "CI should have caught this" theme

Both are gates the repo currently cannot enforce, because the Makefile targets mutate rather
than fail:

  • gofmt drift. CI runs go vet only (ci.yml:29); make fmt rewrites files and never
    fails, so formatting drift can land on main. Peer: ahmetb/kubectx ci.yml:41.
  • go mod tidy drift. No check exists. Makefile:67 runs tidy but mutates, so it can never
    gate CI. Peer: jesseduffield/lazygit ci.yml:146
    go mod tidy && git diff --exit-code || (echo "go.mod file is not clean..." && exit 1).

Both gates print their own remedy via ::error::, which renders as a job annotation in the PR
UI so the contributor sees the fix without opening the log. A bare test -z "$(gofmt -s -l .)"
prints nothing on failure — a red X with an empty log.

I verified both gates actually fire, since a gate that only passes on a clean tree is
satisfied by a step that does nothing:

  • gofmt: added a deliberately unformatted file → exit 1, log names the file and prints
    gofmt -s -w ..
  • tidy: added a source file importing github.com/atotto/clipboard, currently an indirect
    dependency, forcing tidy to promote it → exit 1 with
    + github.com/atotto/clipboard v0.1.4 / - github.com/atotto/clipboard v0.1.4 // indirect.

gofmt -s -l . is currently empty on this tree, so -s adds no churn.

Peer evidence

  • avivsinai/agent-message-queue ci.yml:37Cross-platform build step with
    GOOS=windows go build ./... / GOOS=freebsd go build ./...
  • alexjbarnes/vault-sync .github/workflows/cross-compile.yml:14Cross-compile check
  • docker/model-runner ci.yml:12 — the lint job itself is matrix: goos: [linux, darwin, windows]

Limitations

  • The tidy gate reaches the network. go mod tidy contacts proxy.golang.org (no
    GOFLAGS/GOPROXY/GOPRIVATE is set anywhere in this repo, so defaults apply), so a proxy
    outage would turn a green PR red. ci.yml:33's go install …@latest already has that
    property, so this adds no new class of failure — but it is worth stating rather than
    discovering.
  • I could not run the cross-compile job on self-hosted runners. My fork has no
    self-hosted runners, so ci.yml cannot execute there at all. The job's behaviour is proven
    by the local mutation test above and its cost by your release runs, but the first real
    execution will be the cross-compile check on this PR.
  • The tidy gate leaves the container workspace with a possibly-rewritten go.mod if it passes.
    Nothing downstream in test depends on that, but it is a real side effect.
  • timeout-minutes: 20 on the new job is a guess rather than a measurement, since the job has
    never run. Given 33–38 s legs in production it is ~30× headroom.

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Refactoring (no functional changes)
  • Documentation update
  • CI / Build / Tooling

How Has This Been Tested?

  • Mutation test: injected a Windows-only compile break; confirmed every existing CI gate
    stays green while both Windows legs fail and name the file and line
  • Confirmed fail-fast: false isolates — darwin/arm64 unaffected in the same run
  • Confirmed excluding linux/amd64 leaves no gap (go vet ./... exits 1 on a non-cmd break)
  • Falsified both new gates — each fails on injected drift and prints its remedy
  • Cross-compiled all six targets locally with a cold GOCACHE — all clean today
  • actionlint v1.7.12 — exit 0
  • yaml.safe_load; job/leg/step-ordering assertions via a Python schema check

Checklist

  • My code follows the project's coding style
  • I have performed a self-review of my code
  • I have added tests that prove my fix is effective — n/a, workflow config. Verified by the
    mutation test and gate falsification above.
  • New and existing unit tests pass locally with my changes
  • I have updated the documentation accordingly (not applicable)
  • I have signed the CLA

One question for maintainers

ci.yml has no caching at all — no actions/setup-go, no actions/cache, no
container.volumes — so GOCACHE/GOMODCACHE live in the ephemeral container FS and every run
recompiles the stdlib and every dependency. That already taxes the existing test job, and it is
the honest answer to "why does any of this cost anything". The self-hosted-native fix is small:

    container:
      image: golang:1.26.5
      volumes:
        - /opt/actions-cache/go-build:/root/.cache/go-build
        - /opt/actions-cache/go-mod:/go/pkg/mod

I deliberately left it out of this PR because it needs host-side directories only you control.
Happy to open it separately if that's useful.

AI assistance: Claude Code helped research and verify this change. I reviewed the full diff
and take responsibility for it.

@gemini-code-assist

Copy link
Copy Markdown

Note

Gemini is unable to generate a review for this pull request due to the file types involved not being currently supported.

@coderabbitai

coderabbitai Bot commented Jul 21, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@chethanuk, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 23 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: a3e6a50d-f0ab-4a93-bc5e-0389c31964fb

📥 Commits

Reviewing files that changed from the base of the PR and between c60e886 and dbd878c.

📒 Files selected for processing (1)
  • .github/workflows/ci.yml
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch ci/cross-compile-guard

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chethanuk

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 21, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

CI builds one platform (linux/amd64) while release.yml ships six. The
repo has platform-gated source - shell_windows.go, procattr_windows.go
and their _unix twins - that CI never compiles, so a signature change to
shellCommand or configureProcessGroup compiles green on main and breaks
at tag time, after the tag is public.

Add a cross-compile matrix job covering the five targets the test job
does not, mirroring the shape release.yml:11-29 already runs on this
same runner pool. fail-fast: false so a failure names the target.

The job trusts the workspace the same way the test job does. The
container runs as root over a checkout owned by another uid, so git
reports dubious ownership and Go's VCS stamping - which runs when
building main packages - fails the build with "error obtaining VCS
status: exit status 128". Marking the checkout safe keeps stamping on
and consistent with the test job and the release builds, rather than
papering over it with -buildvcs=false.

Also add two cheap gates the repo lacked: gofmt -s drift (make fmt
rewrites and can never fail CI) and go mod tidy drift (make check
mutates, so it cannot gate either). Both print the remedy.

All six targets build clean today - this is a missing guard, not a fix.
@chethanuk
chethanuk force-pushed the ci/cross-compile-guard branch from 1dd227e to dbd878c Compare July 21, 2026 15:35
@chethanuk

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 21, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant