perf: parallelize and reduce jj/git status subprocess calls - #109
Merged
Conversation
fillCommitMsgFromAncestors used to spawn up to 10 sequential jj log subprocesses, one per generation, looking for the nearest ancestor with a description. heads(ancestors(@, 11) & description(glob:"?*")) finds the same commit in one call.
Status previously ran the working-copy query, head-bookmark lookup, bookmark-list query, and local-ahead count strictly sequentially even though only the second pair depends on the first. Split into fetchWorkingCopyAndHead and fillBookmarkTracking, each running its independent jj calls concurrently via errgroup — cuts the common-case critical path from 4 sequential subprocess round trips to 2.
Status ran `git show-branch --no-name HEAD` and `git log -1 --format=%cd` as two separate subprocesses for two fields of the same commit. `git log -1 --format=%s%x00%cd` gets both in one call, split on git's own %x00 NUL-byte escape so a subject line can't collide with the separator.
git status and git remote don't depend on each other's output but ran sequentially. Backend gains an injectable runGitFn seam (mirroring the jj backend's runJJFn) so Status can dispatch both via errgroup and tests can verify concurrency without real subprocess timing noise.
Pulls in commit_types.perf.bump_patch = true in cog.toml so perf-only PRs (like the preceding backend perf work) trigger an automatic patch release instead of silently landing on main unreleased. Also picks up unrelated upstream template drift: pinned-SHA bumps in .github/workflows/ci.yml and a renovate config refactor (.renovaterc.json now extends go-tools' copier-owned-files preset instead of inlining the same two packageRules).
RunCommand's non-interactive timeout only kills the direct child on ctx cancellation. A subprocess that backgrounds a grandchild (e.g. a shell running "cmd &") can exit itself while that grandchild keeps the inherited stdout/stderr pipe open — cmd.Wait() then blocks draining the pipe for the grandchild's full lifetime regardless of ctx cancellation, a documented os/exec limitation. This is what made CI's TestRunCommand_NonInteractiveTimesOut occasionally take ~60s instead of 50ms. Setting cmd.WaitDelay to the same timeout bounds that drain the same way ctx bounds the process itself.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
jjbackend: collapse the up-to-10-call ancestor-walk (looking for the nearest described commit) into a single revset query.jjbackend: run independentStatussubprocess calls (working-copy/head-bookmark lookups, then bookmark-list/local-ahead counts) concurrently viaerrgroupinstead of sequentially.gitbackend: merge the separate commit-message and commit-timegit logcalls into one, split on git's%x00NUL-byte escape.gitbackend: rungit statusand the remote lookup concurrently (adds an injectablerunGitFnseam mirroring the jj backend'srunJJFn, for deterministic concurrency tests).copier updatefrom go-tools template v1.16.9 → v1.18.0, picking upcommit_types.perf.bump_patch = trueincog.toml(feat(template): treat perf commits as patch-bump-worthy go-tools#62) so this perf-only PR actually triggers a patch release instead of landing onmainunreleased. Also brings unrelated upstream template drift (pinned workflow SHA bumps, a renovate config refactor).internal/backend: boundRunCommand's output-pipe drain withcmd.WaitDelay. A subprocess that backgrounds a grandchild (e.g. a shell runningcmd &) can exit itself while the grandchild keeps the inherited stdout/stderr pipe open;cmd.Wait()then blocks draining that pipe for the grandchild's full lifetime regardless of context cancellation — a documentedos/execlimitation. This is what made CI intermittently take ~60s inTestRunCommand_NonInteractiveTimesOutinstead of 50ms.Test plan
go test ./... -racehk check --allrunJJFn/runGitFn, avoiding real-subprocess timing flakiness)sh -c "sleep 3 &") and assertsRunCommandreturns well under the orphan's lifetimecog bump --auto --dry-runlocally confirms a patch bump (v1.2.2) is now proposed for this commit range🤖 Generated with Claude Code