Skip to content

Don't 500 /sync when an always-included event has no state#19960

Open
ara4n wants to merge 2 commits into
developfrom
matthew/visibility-oob-membership
Open

Don't 500 /sync when an always-included event has no state#19960
ara4n wants to merge 2 commits into
developfrom
matthew/visibility-oob-membership

Conversation

@ara4n

@ara4n ara4n commented Jul 14, 2026

Copy link
Copy Markdown
Member

filter_and_transform_events_for_client computes the requesting user's membership (MSC4115) from the state after each event, fetched only for non-outlier events.

An event let through by always_include_ids therefore has no state to look in if it is flagged as an outlier — which happens when an out-of-band membership event is later de-outliered (we are also in the room and received it over federation) and enters the current state (sync's always_include_ids), while the copy being filtered (e.g. from a worker's not-yet-invalidated cache) still carries the outlier flag.

This hit the 'unreachable' raise, failing the whole /sync with a 500. Return the event without a membership annotation instead (MSC4115 permits omitting it), and downgrade the neighbouring call-invite assert with the same reachability hole to a guard.

Fixes #19858.

Pull Request Checklist

  • Pull request is based on the develop branch
  • Pull request includes a changelog file. The entry should:
    • Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from EventStore to EventWorkerStore.".
    • Use markdown where necessary, mostly for code blocks.
    • End with either a period (.) or an exclamation mark (!).
    • Start with a capital letter.
    • Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry.
  • Code style is correct (run the linters)

filter_and_transform_events_for_client computes the requesting user's
membership (MSC4115) from the state after each event, fetched only for
non-outlier events. An event let through by always_include_ids therefore
has no state to look in if it is flagged as an outlier — which happens
when an out-of-band membership event is later de-outliered (we are also
in the room and received it over federation) and enters the current
state (sync's always_include_ids), while the copy being filtered (e.g.
from a worker's not-yet-invalidated cache) still carries the outlier
flag. This hit the 'unreachable' raise, failing the whole /sync with a
500. Return the event without a membership annotation instead (MSC4115
permits omitting it), and downgrade the neighbouring call-invite assert
with the same reachability hole to a guard.

Fixes #19858.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uk8aPxHn3BHCe52L226jdG
@ara4n
ara4n requested a review from a team as a code owner July 14, 2026 17:40
Comment thread synapse/visibility.py
# invite rejections, invites received over federation, or state
# events needed to authorise other events. None of this applies to
# call invites.
assert state_after_event is not None

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked why fable downgraded the assert into a condition, and:

⏺ Because it's the same landmine one paragraph up from the one we just stepped on — an assert that encodes the belief "state is always present here", where that belief is broken by the same always_include_ids bypass.

The assert's own justification comment says "state_after_event should only be None if the event is an outlier, and earlier code should filter out outliers entirely". That's exactly the invariant #19858 disproved: _check_client_allowed_to_see_event returns outlier-flagged events without state whenever they're in always_include_ids. If a call invite ever arrives at that line with state_after_event=None, the AssertionError 500s the whole request — identical failure mode to the raise we're fixing, just a different event type.

Can it actually happen? Not via the path that bit us — sync's current_state_ids only contains state events, and this branch explicitly checks not event.is_state(). But it's reachable in principle via the other always_include_ids callers: the sticky-events paths (sync.py and sliding_sync/extensions.py) whitelist arbitrary message-like event ids, and sticky events interact with de-outliering too (they're explicitly reconsidered on de-outlier). A sticky m.call.invite with a stale outlier flag would hit the assert.

As for what the guard does when state is missing: the check exists to suppress call invites in public rooms (so they don't ring everyone). With no state we can't read the join rule, so the choices are "deliver anyway" or "suppress on uncertainty". I chose deliver, because an event in always_include_ids was deliberately whitelisted by the caller — silently dropping a whitelisted event on a cache race seemed worse than the rare risk of ringing in a public room.

Comment thread changelog.d/19960.bugfix
@@ -0,0 +1 @@
Fix a bug where `/sync` could return a 500 error ("Missing state for event that is not user's own membership") when an out-of-band membership event, since de-outliered into the room's current state, appeared in a room timeline.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is listed as fixing #19858 which tracks a flake I specifically saw with TestFederationRoomsInvite/Parallel/Remote_invited_user_can_reject_invite_when_homeserver_is_already_participating_in_the_room

From a message you sent in the internal backend room, it looks like you saw this flake with TestFederationRoomsInvite/Parallel/Invited_user_can_reject_invite_over_federation_several_times (different test)

It makes sense that this would probably fix up both as they both involve out-of-band membership (invites over federation)

Comment thread changelog.d/19960.bugfix
@@ -0,0 +1 @@
Fix a bug where `/sync` could return a 500 error ("Missing state for event that is not user's own membership") when an out-of-band membership event, since de-outliered into the room's current state, appeared in a room timeline.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Fix a bug where `/sync` could return a 500 error ("Missing state for event that is not user's own membership") when an out-of-band membership event, since de-outliered into the room's current state, appeared in a room timeline.
Fix a bug where `/sync` could return a 500 error (`Missing state for event that is not user's own membership`) when an out-of-band membership event, since de-outliered into the room's current state, appeared in a room timeline.

Comment thread changelog.d/19960.bugfix
@@ -0,0 +1 @@
Fix a bug where `/sync` could return a 500 error ("Missing state for event that is not user's own membership") when an out-of-band membership event, since de-outliered into the room's current state, appeared in a room timeline.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(we are also in the room and received it over federation) and enters the current state (sync's always_include_ids), while the copy being filtered (e.g. from a worker's not-yet-invalidated cache) still carries the outlier flag.

Would a better fix be to fix the cache problem?

@MadLittleMods
MadLittleMods requested a review from a team July 14, 2026 18:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TestFederationRoomsInvite flake hitting unreachable Exception: Missing state for event that is not user's own membership

2 participants