Skip to content

Fix test_mmap_sparse.py to work on Windows, Mac, and Linux#40

Closed
Copilot wants to merge 1 commit into
new-mmap-testfrom
copilot/develop-mmap-sparse-file-access
Closed

Fix test_mmap_sparse.py to work on Windows, Mac, and Linux#40
Copilot wants to merge 1 commit into
new-mmap-testfrom
copilot/develop-mmap-sparse-file-access

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 22, 2026

test_mmap_sparse.py passed on Linux/Mac but failed on Windows due to two incompatible patterns in Python's mmap on Windows.

Root causes

  • Two mmap objects with different sizes on the same file handle raise OSError (WinError 87)CreateFileMapping rejects a second mapping of a different size from the same handle.
  • Writing via regular file I/O while a write-access (PAGE_READWRITE) mmap is active — conflicts with Windows file-mapping semantics.

Changes

  • Separate "rb" file handle per mmap with explicit access=mmap.ACCESS_READ (PAGE_READONLY). Read-only mappings on Windows allow concurrent writes through other handles without sharing violations.
  • Dedicated "r+b" write handle — mirrors the real-world pattern (background writer process / foreground reader process) and eliminates the same-handle conflict entirely.
  • os.fsync() after each write — ensures the OS page cache is committed before a new mmap is created, needed for correctness on Windows.
  • contextlib.ExitStack — manages all three handles and both mmap objects with safe, exception-proof cleanup.
# Before: single handle, default ACCESS_WRITE, two differently-sized mmaps — fails on Windows
with open(temp_file, "r+b") as f:
    mm1 = mmap.mmap(f.fileno(), len(non_sparse_data), offset=page_size)
    f.write(sparse_region_data)          # write while mmap active — problematic on Windows
    mm2 = mmap.mmap(f.fileno(), len(sparse_region_data), offset=0)  # WinError 87

# After: separate handles, ACCESS_READ, fsync — works on all platforms
with contextlib.ExitStack() as stack:
    read_f1 = stack.enter_context(open(temp_file, "rb"))
    read_f2 = stack.enter_context(open(temp_file, "rb"))
    write_f = stack.enter_context(open(temp_file, "r+b"))
    mm1 = stack.enter_context(mmap.mmap(read_f1.fileno(), len(non_sparse_data),
                                         access=mmap.ACCESS_READ, offset=page_size))
    write_f.write(sparse_region_data); write_f.flush(); os.fsync(write_f.fileno())
    mm2 = stack.enter_context(mmap.mmap(read_f2.fileno(), len(sparse_region_data),
                                         access=mmap.ACCESS_READ, offset=0))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants