Skip to content

feat: recover MLS conversations after key packages refill (WPB-27077) - #4337

Open
sbakhtiarov wants to merge 1 commit into
developfrom
feat/recover-after-key-packages-refill
Open

feat: recover MLS conversations after key packages refill (WPB-27077)#4337
sbakhtiarov wants to merge 1 commit into
developfrom
feat/recover-after-key-packages-refill

Conversation

@sbakhtiarov

@sbakhtiarov sbakhtiarov commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

https://wearezeta.atlassian.net/browse/WPB-27077

https://wearezeta.atlassian.net/browse/WPB-27077

What's new in this PR?

Summary

This PR adds MLS conversation membership recovery for clients that were not added to MLS groups because they had no available key packages or had not yet registered an MLS identity during MLS migration.

After key packages become available and synchronization completes, the client audits its active MLS-capable conversations against CoreCrypto and rejoins any missing groups using external commits.

The implementation keeps normal key-package refill behavior separate from MLS membership-audit orchestration.

Issues

Scenario 1 (MLS):

User has two MLS clients (Client A, Client B)
Client B is offline with no key packages left.
User is added to a new conversation, Client A is added, Client B cannot be added (no key packages).
Client B goes online and refills key packages.

Expected:
Client B joins new MLS conversation

Actual:
Client B cannot receive messages in a new conversation.

Scenario 2 (Proteus -> MLS migration):

User has two Proteus clients (Client A, Client B)
Client B is offline.
Proteus to MLS migration is started.
Existing proteus conversation is migrated to MLS.
Client A creates MLS client, uploads key packages and is added to MLS conversation, Client B cannot be added.
Client B goes online and refills key packages.

Expected:
Client B joins migrated MLS conversation.

Actual:
Client B cannot receive messages in migrated conversation.

Solutions

Durable membership-audit state

Added a user-scoped MLSMembershipAuditRepository backed by the existing Metadata table.
The repository supports three states:

  • NOT_REQUIRED
  • REQUIRED
  • REQUIRED_AFTER_SLOW_SYNC

The marker survives application restarts and is removed only after the complete membership audit succeeds.
The existing "true" metadata value remains compatible with the new state representation. No database schema migration is required.

Key-package depletion detection

RefillKeyPackagesUseCase now treats backend key packages count of zero as a recovery signal.
When zero packages are observed, it:

  • Persists the audit-required marker.
  • Aborts the refill if marker persistence fails.
  • Uploads replacement key packages only after the marker is durable.
  • Leaves the marker set if the upload fails.

RefillKeyPackagesResult.Success now includes:

  • availableCountBeforeRefill
  • refilled
    These values allow the audit flow to determine whether key packages are available after the check:
    refilled == true means this pass successfully uploaded replacements.

availableCountBeforeRefill > 0 handles restart recovery where a previous pass uploaded packages before the application stopped. Normal refills from a low but nonzero count do not independently schedule an audit.

Post-registration recovery

RegisterMLSClientUseCase now persists REQUIRED_AFTER_SLOW_SYNC before registering the MLS client or uploading its initial key packages.
This prevents the audit from starting while registration or its subsequent slow sync is still in progress.
After the manager observes a real post-registration slow-sync cycle completing and incremental sync returning to Live, it promotes the marker to REQUIRED.
This covers existing Proteus clients that receive their MLS identity only after conversations have already migrated.

Dedicated audit orchestration

Added MLSConversationMembershipAuditManager to separate membership recovery from normal key-package management.

The manager:

  1. Observes incremental sync, slow sync, and durable audit state.
  2. Waits until incremental sync is Live and slow sync is complete.
  3. Handles deferred post-registration audit state.
  4. Exposes a force-check signal so a pending audit can wake KeyPackageManager even when incremental sync is already Live.
  5. Verifies that key packages are available before auditing.
  6. Runs the audit in a crypto transaction separate from the key-package refill transaction.
  7. Clears the marker only after a successful audit.
  8. Retains the marker after refill, audit, transaction, or marker-clear failures.

Simplified KeyPackageManager

KeyPackageManager remains focused on its original refill flow:

  1. Wait for incremental sync to become Live.
  2. Verify that MLS is supported.
  3. Verify that this client has registered MLS.
  4. Check whether:
  • the regular 24-hour interval elapsed;
  • the cached count indicates refill is needed; or
  • membership recovery requests a forced check.
  1. Run the refill/check inside an MLS transaction.
  2. Reset the last-check timestamp.
  3. Pass the successful result to the membership-audit manager.
    Normal refill behavior remains independent of slow-sync status. Slow-sync gating applies only to the audit.

Membership audit

Added AuditMLSConversationMembershipUseCase.
The audit queries every active MLS-capable conversation where the self user is still a member:

  • MLS conversations
  • Mixed-protocol conversations
  • All local MLS group states
  • Self conversations

It excludes:

  • Proteus-only conversations
  • Locally deleted conversations
  • Conversations the self user has left

For each conversation, the audit:

  1. Checks whether its MLS group actually exists in CoreCrypto.
  2. Skips groups that already exist.
  3. Rejoins missing groups with external-commit joining enabled.
  4. Continues auditing remaining conversations after individual failures.
  5. Returns overall failure if any conversation could not be checked or repaired.
    The operation is idempotent: subsequent attempts skip groups repaired by an earlier partial audit.

Stale migration metadata recovery

Updated JoinExistingMLSConversationUseCase to handle migrated conversations whose local metadata incorrectly contains:

groupState = ESTABLISHED
epoch = 0

Previously, epoch zero caused the use case to establish a new local group instead of joining the already-existing migrated group.
When external-commit recovery is enabled, the use case now refreshes authoritative conversation metadata before choosing the recovery strategy if the local conversation is:

  • PENDING_AFTER_RESET; or
  • ESTABLISHED at epoch zero.
    After refresh, a migrated conversation with a nonzero backend epoch is joined using an external commit.
    This additional refresh is restricted to external-commit recovery so normal pending Welcome processing remains unchanged.

@github-actions

github-actions Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Test Results

0 tests   0 ✅  0s ⏱️
0 suites  0 💤
0 files    0 ❌

Results for commit 47008d2.

♻️ This comment has been updated with latest results.

@sbakhtiarov
sbakhtiarov force-pushed the feat/recover-after-key-packages-refill branch from b740876 to 47008d2 Compare July 24, 2026 13:09
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.33816% with 20 lines in your changes missing coverage. Please review.
✅ Project coverage is 61.18%. Comparing base (4cde641) to head (47008d2).
⚠️ Report is 1 commits behind head on develop.

Files with missing lines Patch % Lines
...versation/AuditMLSConversationMembershipUseCase.kt 89.58% 3 Missing and 2 partials ⚠️
...versation/MLSConversationMembershipAuditManager.kt 91.52% 2 Missing and 3 partials ⚠️
.../logic/data/conversation/ConversationRepository.kt 0.00% 4 Missing ⚠️
...conversation/JoinExistingMLSConversationUseCase.kt 80.00% 0 Missing and 2 partials ⚠️
...ic/data/keypackage/MLSMembershipAuditRepository.kt 90.47% 0 Missing and 2 partials ⚠️
...om/wire/kalium/logic/feature/client/ClientScope.kt 0.00% 2 Missing ⚠️
Additional details and impacted files
@@              Coverage Diff              @@
##             develop    #4337      +/-   ##
=============================================
+ Coverage      61.09%   61.18%   +0.08%     
- Complexity      4183     4187       +4     
=============================================
  Files           2191     2194       +3     
  Lines          71063    71271     +208     
  Branches        7168     7198      +30     
=============================================
+ Hits           43419    43609     +190     
- Misses         24802    24810       +8     
- Partials        2842     2852      +10     
Files with missing lines Coverage Δ
...um/persistence/dao/conversation/ConversationDAO.kt 86.20% <ø> (ø)
...ersistence/dao/conversation/ConversationDAOImpl.kt 81.01% <100.00%> (+0.15%) ⬆️
...m/logic/feature/client/RegisterMLSClientUseCase.kt 96.96% <100.00%> (+3.86%) ⬆️
...lium/logic/feature/keypackage/KeyPackageManager.kt 96.15% <100.00%> (+16.15%) ⬆️
...ogic/feature/keypackage/RefillKeyPackageUseCase.kt 97.05% <100.00%> (+0.90%) ⬆️
...nc/receiver/conversation/MLSWelcomeEventHandler.kt 83.33% <100.00%> (ø)
...conversation/JoinExistingMLSConversationUseCase.kt 68.06% <80.00%> (-0.05%) ⬇️
...ic/data/keypackage/MLSMembershipAuditRepository.kt 90.47% <90.47%> (ø)
...om/wire/kalium/logic/feature/client/ClientScope.kt 0.00% <0.00%> (ø)
.../logic/data/conversation/ConversationRepository.kt 52.56% <0.00%> (-0.55%) ⬇️
... and 2 more

... and 2 files with indirect coverage changes


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 041f0a8...47008d2. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

Copy link
Copy Markdown
Contributor

🐰 Bencher Report

Branch${GITHUB_HEAD_REF}
Testbedubuntu-latest

⚠️ WARNING: No Threshold found!

Without a Threshold, no Alerts will ever be generated.

Click here to create a new Threshold
For more information, see the Threshold documentation.
To only post results if a Threshold exists, set the --ci-only-thresholds flag.

Click to view all benchmark results
BenchmarkLatencymicroseconds (µs)
com.wire.kalium.benchmarks.logic.CoreLogicBenchmark.createObjectInFiles📈 view plot
⚠️ NO THRESHOLD
600.78 µs
com.wire.kalium.benchmarks.logic.CoreLogicBenchmark.createObjectInMemory📈 view plot
⚠️ NO THRESHOLD
547,955.08 µs
com.wire.kalium.benchmarks.persistence.MessageReadBenchmark.inboxPagingDeepPageBenchmark📈 view plot
⚠️ NO THRESHOLD
93,020.24 µs
com.wire.kalium.benchmarks.persistence.MessageReadBenchmark.inboxPagingFirstPageBenchmark📈 view plot
⚠️ NO THRESHOLD
89,275.79 µs
com.wire.kalium.benchmarks.persistence.MessageReadBenchmark.localMarkAsReadBenchmark📈 view plot
⚠️ NO THRESHOLD
3,732.50 µs
com.wire.kalium.benchmarks.persistence.MessageReadBenchmark.messagePagingDeepPageBenchmark📈 view plot
⚠️ NO THRESHOLD
22,241.30 µs
com.wire.kalium.benchmarks.persistence.MessageReadBenchmark.messagePagingFirstPageBenchmark📈 view plot
⚠️ NO THRESHOLD
11,419.17 µs
com.wire.kalium.benchmarks.persistence.MessagesNoPragmaTuneBenchmark.messageInsertionBenchmark📈 view plot
⚠️ NO THRESHOLD
1,093,138.89 µs
com.wire.kalium.benchmarks.persistence.MessagesNoPragmaTuneBenchmark.queryMessagesBenchmark📈 view plot
⚠️ NO THRESHOLD
20,692.45 µs
🐰 View full continuous benchmarking report in Bencher

@MohamadJaara

Copy link
Copy Markdown
Member

now that i am thinking about it checking every single conversation, it is overkill and not needed, all cases that will cause being in a new MLS group (external link, being added or group reset) will give the conv one of the pending states, so checking those only should do the tirck, wdyt?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants