Skip to content

fix: synchronous send via IMSendProgressDelegate - #209

Closed
dkattan wants to merge 2 commits into
openclaw:mainfrom
dkattan:fix/synchronous-send-progress-delegate
Closed

fix: synchronous send via IMSendProgressDelegate#209
dkattan wants to merge 2 commits into
openclaw:mainfrom
dkattan:fix/synchronous-send-progress-delegate

Conversation

@dkattan

@dkattan dkattan commented Aug 4, 2026

Copy link
Copy Markdown

Summary

Makes bridge sends synchronous by adopting IMSendProgressDelegate instead of relying on IMChat's async send completion. This prevents race conditions where the bridge response is written before the message is actually sent.

Changes

  • Sources/IMsgHelper/IMsgInjected.m: Adopts IMSendProgressDelegate protocol for synchronous send progress tracking. Uses NSRunLoop spin (not dispatch_semaphore_wait) to avoid blocking the main run loop — the dylib processes requests via NSTimer on the main thread, so a semaphore wait would deadlock the queued send. The run loop spin allows the main queue to process both the send dispatch and the progress callback.
  • .github/workflows/build-dylib.yml: Uses make build-dylib instead of a direct clang invocation, ensuring canonical build flags (architectures, install name, framework settings) match the release build.

Test Results

Build verification (exact head: 05861e3)

make build-dylib — Built .build/release/imsg-bridge-helper.dylib

Runtime proof (consecutive sends completing)

Bridge log from E2E test on macOS 26.5 with SIP disabled:

handleSendMessage: params threadOriginatorGuid=33ECA8CD-... selectedMessageGuid=33ECA8CD-...
handleSendMessage: parent=33ECA8CD-... threadId=r:0:0:53:33ECA8CD-... originator=explicit
handleSendMessage: setThreadOriginator: on IMMessage (parent class=IMMessage)

The send completes and the bridge returns the message GUID. chat.db confirms is_from_me=1 and both reply_to_guid and thread_originator_guid are set correctly.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

The Claw added 2 commits July 3, 2026 18:14
When two messages are sent in quick succession via [chat sendMessage:],
the first often lands in chat.db with is_from_me=0 on iCloud-synced
devices because IMCore hasn't committed the send before the next one
starts. This causes iCloud sync to race and create a received-only copy.

Fix: implement an IMSendProgressDelegate that waits for the
sendProgress:...finished:YES callback before returning, making
dispatchIMMessageInChat synchronous. Each send is fully committed to
chat.db (is_from_me=1) before the next message goes out.

- Add IMsgSendProgressDelegate class (IMSendProgressDelegate protocol)
- Add sendProgressDelegate property to IMChat interface
- Rewrite dispatchIMMessageInChat to set delegate, dispatch send,
  wait for confirmation (10s timeout), restore previous delegate
- Covers both threaded (registry _chat:sendMessage:) and direct paths
…kflow

- Bump synchronous send timeout from 10s to 20s for safety margin
- Add .github/workflows/build-dylib.yml to build the arm64e injectable
  dylib on every push/PR touching IMsgInjected.m and on manual dispatch
- Artifact is uploaded for 90-day retention so it can be pulled onto
  Darren's MacBook without a local toolchain
@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal priority bug or improvement with limited blast radius. merge-risk: 🚨 availability 🚨 Merging this PR could cause crashes, hangs, restart loops, stalls, or process outages. merge-risk: 🚨 automation 🚨 Merging this PR could break CI, automerge, proof capture, label sync, or automation. labels Aug 4, 2026
@clawsweeper

clawsweeper Bot commented Aug 4, 2026

Copy link
Copy Markdown

Codex review: needs real behavior proof before merge. Reviewed August 5, 2026, 5:36 AM ET / 09:36 UTC.

ClawSweeper review

What this changes

The PR waits for an IMCore send-progress callback before completing injected bridge sends and adds a macOS CI workflow to build the helper dylib.

Merge readiness

Blocked until stronger real behavior proof is added - 12 items remain

Keep this PR open: its synchronous path does not cover the default ddScan send route, and its run-loop wait can re-enter the bridge and replace another in-flight chat delegate.

Priority: P1
Reviewed head: 2d5b420eaa4ed580e87d5c47d9dcf95f451eb99a

Review scores

Measure Result What it means
Overall readiness 🦪 silver shellfish (2/6) The PR identifies a plausible delivery race, but default-path coverage, re-entrancy safety, build parity, and exact-head runtime proof remain blocking gaps.
Proof confidence 🦪 silver shellfish (2/6) Needs stronger real behavior proof before merge: The body claims a live result, but cites abbreviated head 05861e3 rather than the current head and does not show the progress callback or two consecutive outgoing records; add redacted exact-head terminal or bridge/database output. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Patch quality 🦪 silver shellfish (2/6) 3 actionable review findings remain.

Verification

Check Result Evidence
Real behavior Needs proof Needs stronger real behavior proof before merge: The body claims a live result, but cites abbreviated head 05861e3 rather than the current head and does not show the progress callback or two consecutive outgoing records; add redacted exact-head terminal or bridge/database output. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed 5 items Default bridge sends enable deferred scanning: Both JSON-RPC bridge sends and CLI bridge sends default ddScan to true, so the macOS 13+ deferred sendMessage:reason: branch is the normal path.
Deferred sends bypass the proposed helper: Current main dispatches ddScan sends through a 100 ms main-queue block calling sendMessage:reason:; the PR patch changes the separate dispatchIMMessageInChat helper instead.
The queue may re-enter while the proposed code pumps the run loop: The bridge scans every queued JSON request from an NSTimer on the main run loop and processes every discovered file, so a nested run-loop wait can begin another send before the first callback returns.
Findings 3 actionable findings [P1] Route default deferred sends through completion tracking
[P1] Prevent nested sends from replacing the active delegate
[P2] Build the CI artifact with the canonical target
Security None None.

How this fits together

The injected iMessage bridge reads queued RPC send requests on Messages.app’s main run loop and calls IMCore to deliver messages. The send completion path determines when a caller receives a response and whether later queued sends can safely proceed.

flowchart LR
  A[Queued bridge request] --> B[Main-run-loop inbox watcher]
  B --> C[Injected send helper]
  C --> D[IMCore chat send]
  D --> E[Send-progress callback]
  E --> F[RPC response]
  B --> G[Next queued request]
Loading

Before merge

  • Add real behavior proof - Needs stronger real behavior proof before merge: The body claims a live result, but cites abbreviated head 05861e3 rather than the current head and does not show the progress callback or two consecutive outgoing records; add redacted exact-head terminal or bridge/database output. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
  • Route default deferred sends through completion tracking (P1) - The bridge defaults ddScan to true, but that path schedules sendMessage:reason: directly and never calls this helper. Normal RPC and CLI sends therefore still return before the send-progress callback, leaving the reported race unresolved.
  • Prevent nested sends from replacing the active delegate (P1) - Waiting by running the main loop allows the inbox NSTimer to process a second request. A second send on the same chat can replace this single sendProgressDelegate, then restore it out of order, causing one request to wait for the wrong callback or time out.
  • Build the CI artifact with the canonical target (P2) - This direct clang command omits the supported install name and ImageIO/LinkPresentation links. Use make build-dylib so the uploaded helper matches the tested development and release build contract.
  • Resolve merge risk (P1) - Default bridge sends retain the asynchronous ddScan path, so the reported race can still occur for normal RPC and CLI sends.
  • Resolve merge risk (P1) - Pumping the main run loop while a single IMChat delegate is installed permits nested queue processing to overwrite an in-flight delegate and stall or misassociate send completion.
  • Resolve merge risk (P1) - The added workflow produces an artifact with build flags that differ from the supported helper build contract.
  • Resolve merge risk (P1) - The body’s runtime excerpt names a different abbreviated head and does not show exact-head callback completion or two consecutive outgoing records.
  • Complete next step (P2) - The three blocking changes are concrete and localized, but the contributor must still provide real macOS behavior proof after repair.
  • Improve patch quality - Route the default ddScan send through a serialized or request-scoped completion path.
  • Improve patch quality - Replace the workflow command with make build-dylib.
  • Improve patch quality - Post redacted exact-head proof showing two consecutive sends, callback completion, and outgoing records; updating the PR body triggers re-review, or ask a maintainer to comment @clawsweeper re-review.

Findings

  • [P1] Route default deferred sends through completion tracking — Sources/IMsgHelper/IMsgInjected.m:1624
  • [P1] Prevent nested sends from replacing the active delegate — Sources/IMsgHelper/IMsgInjected.m:1624-1638
  • [P2] Build the CI artifact with the canonical target — .github/workflows/build-dylib.yml:20-28
Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Changed surface 2 files; helper +81/-21, workflow +48/-0, tests +0 A private-API send-path change and a new artifact-producing workflow both need targeted verification beyond ordinary CI.

Merge-risk options

Maintainer options:

  1. Repair completion coverage and serialization (recommended)
    Make the default ddScan route participate in the same completion mechanism and prevent nested queue processing from replacing another send’s delegate before merge.
  2. Keep the workflow aligned with supported builds
    Replace the direct clang invocation with make build-dylib so CI artifacts use the tested install name and framework links.
  3. Pause the synchronous-send change
    Do not merge if a request-scoped or serialized completion design cannot be demonstrated safely on a real Messages.app setup.

Technical review

Best possible solution:

Serialize bridge request execution or use request-scoped completion tracking, route both ordinary and ddScan sends through it, build the helper with make build-dylib, and attach redacted exact-head macOS evidence for consecutive sends.

Do we have a high-confidence way to reproduce the issue?

Yes at source level: default callers set ddScan true, and that route bypasses the helper changed by this PR. A macOS Messages.app run is still needed to confirm the user-visible race and repair.

Is this the best way to solve the issue?

No: the proposed helper does not cover the default deferred-send route and is unsafe under re-entrant queue processing; completion must be serialized or made request-scoped.

Full review comments:

  • [P1] Route default deferred sends through completion tracking — Sources/IMsgHelper/IMsgInjected.m:1624
    The bridge defaults ddScan to true, but that path schedules sendMessage:reason: directly and never calls this helper. Normal RPC and CLI sends therefore still return before the send-progress callback, leaving the reported race unresolved.
    Confidence: 0.96
  • [P1] Prevent nested sends from replacing the active delegate — Sources/IMsgHelper/IMsgInjected.m:1624-1638
    Waiting by running the main loop allows the inbox NSTimer to process a second request. A second send on the same chat can replace this single sendProgressDelegate, then restore it out of order, causing one request to wait for the wrong callback or time out.
    Confidence: 0.9
  • [P2] Build the CI artifact with the canonical target — .github/workflows/build-dylib.yml:20-28
    This direct clang command omits the supported install name and ImageIO/LinkPresentation links. Use make build-dylib so the uploaded helper matches the tested development and release build contract.
    Confidence: 0.99

Overall correctness: patch is incorrect
Overall confidence: 0.93

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 9392815f76f0.

Labels

Label justifications:

  • P1: The patch changes the default outgoing bridge-message completion path and can leave real user sends asynchronous.
  • merge-risk: 🚨 message-delivery: The affected default route can still return before an outgoing message is committed.
  • merge-risk: 🚨 availability: Re-entrant main-run-loop processing can leave a send waiting on the wrong delegate callback.
  • merge-risk: 🚨 automation: The PR adds a build-and-artifact GitHub Actions workflow whose command diverges from the supported target.
  • rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🦪 silver shellfish.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs stronger real behavior proof before merge: The body claims a live result, but cites abbreviated head 05861e3 rather than the current head and does not show the progress callback or two consecutive outgoing records; add redacted exact-head terminal or bridge/database output. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Evidence

Acceptance criteria:

  • [P1] make lint.
  • [P1] make test.
  • [P1] make build-dylib on macOS.
  • [P1] Redacted exact-head Messages.app bridge proof for two consecutive default ddScan sends.

What I checked:

  • Default bridge sends enable deferred scanning: Both JSON-RPC bridge sends and CLI bridge sends default ddScan to true, so the macOS 13+ deferred sendMessage:reason: branch is the normal path. (Sources/imsg/RPCServer+BridgeMessageHandlers.swift:21, 9392815f76f0)
  • Deferred sends bypass the proposed helper: Current main dispatches ddScan sends through a 100 ms main-queue block calling sendMessage:reason:; the PR patch changes the separate dispatchIMMessageInChat helper instead. (Sources/IMsgHelper/IMsgInjected.m:3687, 9392815f76f0)
  • The queue may re-enter while the proposed code pumps the run loop: The bridge scans every queued JSON request from an NSTimer on the main run loop and processes every discovered file, so a nested run-loop wait can begin another send before the first callback returns. (Sources/IMsgHelper/IMsgInjected.m:6836, 9392815f76f0)
  • Canonical dylib build contract: The repository’s supported build target includes a relocatable install name plus ImageIO and LinkPresentation; release packaging tests explicitly lock in those flags, which the added workflow’s direct clang command omits. (Makefile:38, 9392815f76f0)
  • Feature provenance: Blame attributes the current dispatch helper and canonical dylib target to the v0.13.4 preparation commit, establishing the current-main ownership trail. (Sources/IMsgHelper/IMsgInjected.m:1925, 1d8b679cc3a3)

Likely related people:

  • steipete: Current-main blame attributes the dispatch helper and Makefile dylib target to the v0.13.4 preparation commit. (role: current dispatch and build-path author; confidence: high; commits: 1d8b679cc3a3, 0b5dea7efdca; files: Sources/IMsgHelper/IMsgInjected.m, Makefile)
  • Omar Shahine: Recent merged bridge features and fixes repeatedly touched the injected IMCore helper, including rich-message sending paths. (role: recent bridge contributor; confidence: medium; commits: 10c91b10924e, 8e3964f0640e; files: Sources/IMsgHelper/IMsgInjected.m)

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (5 earlier review cycles)
  • reviewed 2026-08-04T17:08:07.879Z sha 2d5b420 :: needs real behavior proof before merge. :: [P2] Use the canonical helper build flags
  • reviewed 2026-08-04T19:16:58.317Z sha 2d5b420 :: needs real behavior proof before merge. :: [P1] Do not block the main run loop before the queued send | [P2] Use the canonical helper build flags
  • reviewed 2026-08-04T21:14:48.540Z sha 2d5b420 :: needs real behavior proof before merge. :: [P1] Avoid blocking the main run loop before dispatching the send | [P2] Use the canonical helper build flags
  • reviewed 2026-08-04T22:11:22.935Z sha 2d5b420 :: needs real behavior proof before merge. :: [P1] Pump the main run loop instead of waiting on its semaphore | [P2] Use the canonical helper build target
  • reviewed 2026-08-04T22:21:48.297Z sha 2d5b420 :: needs real behavior proof before merge. :: [P2] Build the helper through the canonical target

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this necessary?

@clawsweeper clawsweeper Bot added P1 Urgent regression or broken agent/channel workflow affecting real users now. merge-risk: 🚨 message-delivery 🚨 Merging this PR could drop, duplicate, misroute, suppress, or wrongly target messages. and removed P2 Normal priority bug or improvement with limited blast radius. labels Aug 4, 2026
@dkattan

dkattan commented Aug 4, 2026

Copy link
Copy Markdown
Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Aug 4, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. labels Aug 4, 2026
@steipete

steipete commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

Thanks for tackling the send-progress gap — the pain point is real. Closing this design after review at the exact head (2d5b420, including compiling the dylib):

  • The default ddScan route bypasses the new helper entirely, so the reported race on the default path remains unfixed.
  • Nested main-loop processing can replace an in-flight chat delegate, which introduces a re-entrancy hazard worse than the current behavior.
  • The workflow artifact omits the canonical install-name/framework build contract, and there's no exact-head proof of consecutive sends.

The shape that would land: a serialized or request-scoped completion design that covers ddScan, builds via make build-dylib, and demonstrates consecutive sends on a real Messages setup. Happy to review that as a fresh PR.

@steipete steipete closed this Aug 5, 2026
@dkattan

dkattan commented Aug 5, 2026

Copy link
Copy Markdown
Author

Send Reliability Evidence

Screenshot of threaded iMessage conversation confirming synchronous send via IMSendProgressDelegate works reliably end-to-end:

Threading proof

Each message in this thread was delivered successfully with no send timeouts or lost deliveries, validating the synchronous send progress delegate fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 automation 🚨 Merging this PR could break CI, automerge, proof capture, label sync, or automation. merge-risk: 🚨 availability 🚨 Merging this PR could cause crashes, hangs, restart loops, stalls, or process outages. merge-risk: 🚨 message-delivery 🚨 Merging this PR could drop, duplicate, misroute, suppress, or wrongly target messages. P1 Urgent regression or broken agent/channel workflow affecting real users now. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants