Clamp post-commit free horizon to the savepoint purge horizon#1296
Merged
Conversation
An ephemeral Savepoint can be dropped from a different thread than the one committing the WriteTransaction that created it, since WriteTransaction is Sync. During a durable commit, flush_data_allocated_pages() purges DATA_ALLOCATED_TABLE down to the oldest savepoint's transaction, keeping the entries that savepoint pins. The post-commit epilogue then frees DATA_FREED_TABLE pages up to a horizon derived from the live read transactions. A savepoint is also a live read, so absent concurrency the two horizons agree. If the ephemeral Savepoint::drop lands between the purge and the epilogue, it removes the savepoint's read reference, so the epilogue's horizon advances past the savepoint the purge accounted for. The epilogue then frees pages that the surviving DATA_ALLOCATED_TABLE entries still name, so check_integrity() hits the assertion in check_repaired_allocated_pages_table (a debug-build panic / false corruption report). This is the concurrent-drop sibling of the staged-deletion bug fixed in d098d3b. No durable corruption is reachable: the next durable commit re-purges the stale entries. Capture the purge horizon in flush_data_allocated_pages() and clamp the epilogue's free horizon to it. The clamp is a no-op in every non-racy case; a dropped savepoint's entries are simply reclaimed by the next durable commit. Assisted-by: Claude Code https://claude.ai/code/session_019rKhoFerkDeoUn1DqpJn3t
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1296 +/- ##
==========================================
- Coverage 90.21% 90.21% -0.01%
==========================================
Files 36 36
Lines 16318 16326 +8
==========================================
+ Hits 14722 14728 +6
- Misses 1596 1598 +2
🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
WriteTransactionisSync, so an ephemeralSavepointcan be dropped from a differentthread than the one committing the transaction that created it. During a durable commit:
flush_data_allocated_pages()purgesDATA_ALLOCATED_TABLEdown to the oldest savepoint'stransaction, keeping the entries that savepoint pins.
process_data_freed_pages_after_commit()) freesDATA_FREED_TABLEpages up to a horizon derived from the live read transactions.A savepoint is also a live read, so absent concurrency the two horizons agree. But if the
ephemeral
Savepoint::droplands between the purge and the epilogue, it removes thesavepoint's read reference and the epilogue's horizon advances past the savepoint the purge
accounted for. The epilogue then frees pages that surviving
DATA_ALLOCATED_TABLEentriesstill name. This is the concurrent-drop sibling of the staged-deletion bug fixed in d098d3b.
Impact
check_integrity()hits theassert!incheck_repaired_allocated_pages_table, i.e. adebug-build panic / false corruption report. No durable corruption is reachable: the stale
entries are always for transactions strictly older than any restorable savepoint (the drop
only advances the horizon when the dropped savepoint was the sole oldest live read, so the
over-freed pages are older than the next oldest read, hence older than every surviving
savepoint), and
restore_savepoint()only readsDATA_ALLOCATED_TABLEentries newer than therestored savepoint. The next durable commit re-purges the stale entries regardless.
Fix
Return the purge horizon from
flush_data_allocated_pages()and clamp the epilogue's freehorizon to it. The clamp is a no-op in every non-racy case (a live savepoint always bounds
oldest_live_read_transaction()); a concurrently-dropped savepoint's entries are simplyreclaimed by the next durable commit.
Testing
check_integrity_ephemeral_savepoint_drop_racing_commituses the existing
BlockingSyncBackendto pause the commit at its fsync (between the purgeand the epilogue), drops the savepoint in that window, then asserts
check_integrity(). Itpanics at
check_repaired_allocated_pages_tablewithout the fix and passes with it.cargo test --all-features,cargo fmt --check, andcargo clippy --all-targets --all-featuresare all clean.🤖 Generated with Claude Code
https://claude.ai/code/session_019rKhoFerkDeoUn1DqpJn3t
Generated by Claude Code