Skip to content

fix(atomic-io): retry rename on transient Windows EPERM/EBUSY - #40

Open
DrChrisWagner wants to merge 1 commit into
FerroxLabs:mainfrom
DrChrisWagner:fix/atomic-rename-windows-eperm
Open

fix(atomic-io): retry rename on transient Windows EPERM/EBUSY#40
DrChrisWagner wants to merge 1 commit into
FerroxLabs:mainfrom
DrChrisWagner:fix/atomic-rename-windows-eperm

Conversation

@DrChrisWagner

Copy link
Copy Markdown

Summary

Harden atomic file writes against the intermittent Windows EPERM/EBUSY that occurs when antivirus or the Search indexer momentarily holds a handle on a rename target. This surfaced in the wild as the dream cycle failing to persist .ijfw/.dream-state-v2.json:

wiki-compile: EPERM: operation not permitted, rename
  '...\.ijfw\.dream-state-v2.json.tmp' -> '...\.ijfw\.dream-state-v2.json'

Root cause

Two independent weaknesses with the same failure mode:

  1. src/dream/state-file.js writeDreamState() rolled its own write with a fixed tmp name (p + '.tmp') and a bare renameSync with no retry. The fixed name collides across concurrent runs and is blocked by a stale .tmp left by a crashed run; the un-retried rename fails on Windows whenever the destination is transiently locked.

  2. src/lib/atomic-io.js writeAtomic() uses a randomized tmp name (good) but its renameSync likewise had no Windows retry — the same latent bug for every caller of the shared helper.

POSIX rename(2) is atomic and does not exhibit this; the problem is Windows/NTFS-specific.

Changes

  • src/lib/atomic-io.js: add renameWithRetry() — bounded exponential backoff (10ms → 250ms cap, max 10 attempts), retries only the known-transient codes (EPERM/EACCES/EBUSY/EEXIST) and rethrows everything else immediately so real errors are never masked. On non-Windows it does a single renameSyncPOSIX behavior is unchanged. writeAtomic() now calls it instead of renameSync.
  • src/dream/state-file.js: writeDreamState() now delegates to the shared writeAtomic() (randomized tmp name + the new rename-retry), replacing its bespoke fixed-tmp + renameSync.

Testing

  • Existing suites pass with no regressions: test-atomic-io.js, test-dream-state-file.js, test-tmp-suffix.js.
  • Concurrency hammer on Windows: 500 rapid sequential rewrites + a 50-way concurrent burst against the real dream-state file — 0 failures, file always valid JSON after every write.

Notes

Reproduced and fixed on Windows 11 / Node 22. The fix is defensive and no-op on POSIX, so it should be safe across all supported platforms.

Rename-over-existing on NTFS intermittently fails with EPERM/EACCES/EBUSY
when antivirus or the Search indexer momentarily holds a handle on the
target. This surfaced as the dream cycle failing to persist
.ijfw/.dream-state-v2.json:

  EPERM: operation not permitted, rename
    '...\.dream-state-v2.json.tmp' -> '...\.dream-state-v2.json'

Two independent weaknesses with the same failure mode:
- src/dream/state-file.js writeDreamState() used a fixed tmp name plus a
  bare renameSync with no retry (fixed name also collides across
  concurrent runs and is blocked by a stale .tmp from a crashed run).
- src/lib/atomic-io.js writeAtomic() used a randomized tmp name but its
  renameSync likewise had no Windows retry - the same latent bug for
  every caller of the shared helper.

Add renameWithRetry() to atomic-io: bounded exponential backoff, retries
only the known-transient codes (EPERM/EACCES/EBUSY/EEXIST) and rethrows
everything else immediately so real errors are not masked. On non-Windows
it does a single renameSync, so POSIX behavior is unchanged. Route
writeAtomic through it, and make writeDreamState delegate to the shared
writeAtomic instead of its bespoke fixed-tmp + renameSync.

Tests: test-atomic-io / test-dream-state-file / test-tmp-suffix pass with
no regressions; a 550-write concurrent hammer (Win11 / Node 22) against
the real dream-state file had 0 failures and the file stayed valid JSON.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant