Skip to content

fix: update data_substrate — prevent dirty_data_key_count_ underflow - #458

Closed
githubzilla wants to merge 1 commit into
eloqdata:mainfrom
githubzilla:fix/backfill-dirty-key-count-underflow
Closed

fix: update data_substrate — prevent dirty_data_key_count_ underflow#458
githubzilla wants to merge 1 commit into
eloqdata:mainfrom
githubzilla:fix/backfill-dirty-key-count-underflow

Conversation

@githubzilla

@githubzilla githubzilla commented Mar 18, 2026

Copy link
Copy Markdown
Collaborator

Summary

Update data_substrate submodule to include fix for intermittent assertion failure:

Assertion `dirty_delta >= 0 || dirty_data_key_count_ >= static_cast<size_t>(-dirty_delta)' failed.

Three bugs fixed in tx_service (see eloqdata/tx_service#465 for full details):

  1. CleanBucketData freeing entries being checkpointedCanBeCleaned now checks !GetBeingCkpt() to prevent dirty count double-decrement (confirmed root cause via core dump + diagnostic logs)
  2. BackFill wrong operation order — reordered SetCommitTsPayloadStatus before SetCkptTs to prevent flush bit from being cleared; added missing OnCommittedUpdate in BackFill and ReadOutsideCc paths
  3. ClusterConfigCcMap missing OnCommittedUpdate — added dirty count tracking at both SetCommitTsPayloadStatus sites

Testing

Verified by running cluster_scale_test.py (all 5 tests) dozens of times with no assertion failures.

Summary by CodeRabbit

  • Chores
    • Updated an internal submodule/dependency reference to a newer revision. No functional or public API changes; end-user behavior remains unchanged.

@coderabbitai

coderabbitai Bot commented Mar 18, 2026

Copy link
Copy Markdown
Contributor

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Updated the data_substrate git submodule pointer from ce06eee580f6bd35aaf1e8caed8b31286d11df59 to 807e7078303c3037c1bda92ee8865401e1d84cf5; no other code or API changes. Lines changed: +1 / -1.

Changes

Cohort / File(s) Summary
Submodule Update
./.gitmodules, data_substrate
Changed data_substrate submodule checked-out commit from ce06eee580f6bd35aaf1e8caed8b31286d11df59807e7078303c3037c1bda92ee8865401e1d84cf5 (metadata-only change).

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~2 minutes

Possibly related PRs

Suggested reviewers

  • liunyl
  • thweetkomputer

Poem

🐇 I nudged a hash beneath the log,
A tiny hop, a quiet cog,
The pointer moved, the code sleeps tight,
I twitch my whiskers in the night,
Repo dreams on in pale moonlight.

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main purpose of the PR: updating the data_substrate submodule to fix a specific bug related to dirty_data_key_count_ underflow.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@githubzilla
githubzilla force-pushed the fix/backfill-dirty-key-count-underflow branch from ec5f92a to da92602 Compare March 18, 2026 15:06
@githubzilla githubzilla changed the title chore: update data_substrate — fix BackFill dirty_data_key_count_ underflow fix: update data_substrate — prevent dirty_data_key_count_ underflow Mar 21, 2026
@githubzilla
githubzilla force-pushed the fix/backfill-dirty-key-count-underflow branch from 51d2453 to 9831b0f Compare March 21, 2026 03:31
…erflow

Update data_substrate submodule to include the fix for an intermittent
assertion failure in AdjustDataKeyStats where dirty_data_key_count_
underflows during cluster scale-out.

See eloqdata/tx_service#465 for details.
@githubzilla
githubzilla force-pushed the fix/backfill-dirty-key-count-underflow branch from 6cd7387 to 49f0dcf Compare March 24, 2026 04:51
@liunyl liunyl closed this Jun 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants