feat(dm): one moss node for every conversation - #37
Merged
Conversation
Each session started its own moss node. Node identity is per process — the
keystore is a process global — so N open conversations meant N nodes presenting
the SAME peer id from N ports. A remote peer keeps one session per identity: it
closed the rest on arrival and declined to dial the others at all, because it
already held that id.
Measured across three clients over three days: 33,715 sessions, 32,330 of them
dead inside one second (95%), sustained at 205 / 131 / 113 discarded Noise
handshakes an hour against only 6-8 distinct peers each. One identity was seen
on 27 different ports within a single hour. That is what users experienced as a
chat that only worked once every other chat was closed.
Sessions now share one node (refcounted exactly like the relay node) and
separate themselves by room, using moss v0.8.19's per-room subscribe/publish.
A joined room is byte-identical to the same room owned outright, so this stays
wire-compatible with every already-released client.
Three things the shared node forced into the open:
- `unsubscribe` was never bound. The FFI has exposed Moss_Unsubscribe all
along; mosh's `unsubscribe_voice_call` was a stub returning Ok(()). With a
node per session it did not matter, because dropping the node ended its
subscriptions. Now a closed conversation must say so, or it keeps
receiving forever. Bound, and close_session leaves the room.
- `publish_room` needed the same "no peers yet is not a failure" tolerance as
`publish`, or every message sent before the mesh forms is marked Failed.
- `mesh_info` reports the whole node, so one chat's diagnostics listed every
other chat's channels. Filtered to the session's own. Peer lists stay whole
on purpose — has_live_peer matches the counterpart by id against them.
Verified on live moss: attachment transfer and voice call E2E both pass on the
shared node, plus 187 unit tests and clippy -D warnings.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Each session started its own moss node. Node identity is per process (the keystore is a process global), so N open conversations meant N nodes presenting the same peer id from N ports. A remote peer keeps one session per identity: it closed the rest on arrival, and declined to dial the others at all because it already held that id.
Measured over three days, mosh clients only:
95%+ of sessions dead inside a second, against 6–8 distinct peers each. One identity on 27 ports inside one hour. That is what users hit as "the chat only works once I close every other chat".
What changed
Sessions share one node, refcounted exactly like the existing relay node (
ensure_relay_up/release_relay), and separate themselves by room using moss v0.8.19's per-room subscribe/publish. A joined room is byte-identical to the same room owned outright, so this stays wire-compatible with every already-released client.Three things the shared node forced into the open:
unsubscribewas never bound.Moss_Unsubscribehas existed in the FFI all along;unsubscribe_voice_callwas a stub returningOk(()). With a node per session that was harmless — dropping the node ended its subscriptions. Now a closed conversation must say so or it keeps receiving forever. Bound, andclose_sessionleaves the room.publish_roomneededpublish's tolerance for "no peers yet", or every message sent before the mesh forms is marked Failed. (Caught by the existing suite.)mesh_inforeports the whole node, so one chat's diagnostics listed every other chat's channels. Filtered to the session's own; peer lists stay whole becausehas_live_peermatches the counterpart by id against them.Verification
sessions_share_one_node_and_close_releases_it: two conversations, one node, separate rooms; closing one keeps the node up, closing the last takes it down.clippy --all-targets -D warnings,src-taurichecks.Moss symbol unavailable: Moss_JoinRoom), verified against the released v0.8.18 dll — not a crash.Known, not caused here
private_dm_runtime_exchanges_e2ee_message_over_moss(#[ignore]d) fails. Verified it fails identically on unchangedmainand against the v0.8.18 library, so it predates this work: both test runtimes drain one process-global inbound queue, and whichever polls first consumes the delivery ack. Left alone rather than folded into this change.