fix(atomic-io): retry rename on transient Windows EPERM/EBUSY - #40
Open
DrChrisWagner wants to merge 1 commit into
Open
fix(atomic-io): retry rename on transient Windows EPERM/EBUSY#40DrChrisWagner wants to merge 1 commit into
DrChrisWagner wants to merge 1 commit into
Conversation
Rename-over-existing on NTFS intermittently fails with EPERM/EACCES/EBUSY
when antivirus or the Search indexer momentarily holds a handle on the
target. This surfaced as the dream cycle failing to persist
.ijfw/.dream-state-v2.json:
EPERM: operation not permitted, rename
'...\.dream-state-v2.json.tmp' -> '...\.dream-state-v2.json'
Two independent weaknesses with the same failure mode:
- src/dream/state-file.js writeDreamState() used a fixed tmp name plus a
bare renameSync with no retry (fixed name also collides across
concurrent runs and is blocked by a stale .tmp from a crashed run).
- src/lib/atomic-io.js writeAtomic() used a randomized tmp name but its
renameSync likewise had no Windows retry - the same latent bug for
every caller of the shared helper.
Add renameWithRetry() to atomic-io: bounded exponential backoff, retries
only the known-transient codes (EPERM/EACCES/EBUSY/EEXIST) and rethrows
everything else immediately so real errors are not masked. On non-Windows
it does a single renameSync, so POSIX behavior is unchanged. Route
writeAtomic through it, and make writeDreamState delegate to the shared
writeAtomic instead of its bespoke fixed-tmp + renameSync.
Tests: test-atomic-io / test-dream-state-file / test-tmp-suffix pass with
no regressions; a 550-write concurrent hammer (Win11 / Node 22) against
the real dream-state file had 0 failures and the file stayed valid JSON.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Harden atomic file writes against the intermittent Windows
EPERM/EBUSYthat occurs when antivirus or the Search indexer momentarily holds a handle on a rename target. This surfaced in the wild as the dream cycle failing to persist.ijfw/.dream-state-v2.json:Root cause
Two independent weaknesses with the same failure mode:
src/dream/state-file.jswriteDreamState()rolled its own write with a fixed tmp name (p + '.tmp') and a barerenameSyncwith no retry. The fixed name collides across concurrent runs and is blocked by a stale.tmpleft by a crashed run; the un-retried rename fails on Windows whenever the destination is transiently locked.src/lib/atomic-io.jswriteAtomic()uses a randomized tmp name (good) but itsrenameSynclikewise had no Windows retry — the same latent bug for every caller of the shared helper.POSIX
rename(2)is atomic and does not exhibit this; the problem is Windows/NTFS-specific.Changes
src/lib/atomic-io.js: addrenameWithRetry()— bounded exponential backoff (10ms → 250ms cap, max 10 attempts), retries only the known-transient codes (EPERM/EACCES/EBUSY/EEXIST) and rethrows everything else immediately so real errors are never masked. On non-Windows it does a singlerenameSync— POSIX behavior is unchanged.writeAtomic()now calls it instead ofrenameSync.src/dream/state-file.js:writeDreamState()now delegates to the sharedwriteAtomic()(randomized tmp name + the new rename-retry), replacing its bespoke fixed-tmp +renameSync.Testing
test-atomic-io.js,test-dream-state-file.js,test-tmp-suffix.js.Notes
Reproduced and fixed on Windows 11 / Node 22. The fix is defensive and no-op on POSIX, so it should be safe across all supported platforms.