Don't 500 /sync when an always-included event has no state#19960
Conversation
filter_and_transform_events_for_client computes the requesting user's membership (MSC4115) from the state after each event, fetched only for non-outlier events. An event let through by always_include_ids therefore has no state to look in if it is flagged as an outlier — which happens when an out-of-band membership event is later de-outliered (we are also in the room and received it over federation) and enters the current state (sync's always_include_ids), while the copy being filtered (e.g. from a worker's not-yet-invalidated cache) still carries the outlier flag. This hit the 'unreachable' raise, failing the whole /sync with a 500. Return the event without a membership annotation instead (MSC4115 permits omitting it), and downgrade the neighbouring call-invite assert with the same reachability hole to a guard. Fixes #19858. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uk8aPxHn3BHCe52L226jdG
| # invite rejections, invites received over federation, or state | ||
| # events needed to authorise other events. None of this applies to | ||
| # call invites. | ||
| assert state_after_event is not None |
There was a problem hiding this comment.
I checked why fable downgraded the assert into a condition, and:
⏺ Because it's the same landmine one paragraph up from the one we just stepped on — an assert that encodes the belief "state is always present here", where that belief is broken by the same always_include_ids bypass.
The assert's own justification comment says "state_after_event should only be None if the event is an outlier, and earlier code should filter out outliers entirely". That's exactly the invariant #19858 disproved: _check_client_allowed_to_see_event returns outlier-flagged events without state whenever they're in always_include_ids. If a call invite ever arrives at that line with state_after_event=None, the AssertionError 500s the whole request — identical failure mode to the raise we're fixing, just a different event type.
Can it actually happen? Not via the path that bit us — sync's current_state_ids only contains state events, and this branch explicitly checks not event.is_state(). But it's reachable in principle via the other always_include_ids callers: the sticky-events paths (sync.py and sliding_sync/extensions.py) whitelist arbitrary message-like event ids, and sticky events interact with de-outliering too (they're explicitly reconsidered on de-outlier). A sticky m.call.invite with a stale outlier flag would hit the assert.
As for what the guard does when state is missing: the check exists to suppress call invites in public rooms (so they don't ring everyone). With no state we can't read the join rule, so the choices are "deliver anyway" or "suppress on uncertainty". I chose deliver, because an event in always_include_ids was deliberately whitelisted by the caller — silently dropping a whitelisted event on a cache race seemed worse than the rare risk of ringing in a public room.
| @@ -0,0 +1 @@ | |||
| Fix a bug where `/sync` could return a 500 error ("Missing state for event that is not user's own membership") when an out-of-band membership event, since de-outliered into the room's current state, appeared in a room timeline. | |||
There was a problem hiding this comment.
This is listed as fixing #19858 which tracks a flake I specifically saw with TestFederationRoomsInvite/Parallel/Remote_invited_user_can_reject_invite_when_homeserver_is_already_participating_in_the_room
From a message you sent in the internal backend room, it looks like you saw this flake with TestFederationRoomsInvite/Parallel/Invited_user_can_reject_invite_over_federation_several_times (different test)
It makes sense that this would probably fix up both as they both involve out-of-band membership (invites over federation)
| @@ -0,0 +1 @@ | |||
| Fix a bug where `/sync` could return a 500 error ("Missing state for event that is not user's own membership") when an out-of-band membership event, since de-outliered into the room's current state, appeared in a room timeline. | |||
There was a problem hiding this comment.
| Fix a bug where `/sync` could return a 500 error ("Missing state for event that is not user's own membership") when an out-of-band membership event, since de-outliered into the room's current state, appeared in a room timeline. | |
| Fix a bug where `/sync` could return a 500 error (`Missing state for event that is not user's own membership`) when an out-of-band membership event, since de-outliered into the room's current state, appeared in a room timeline. |
| @@ -0,0 +1 @@ | |||
| Fix a bug where `/sync` could return a 500 error ("Missing state for event that is not user's own membership") when an out-of-band membership event, since de-outliered into the room's current state, appeared in a room timeline. | |||
There was a problem hiding this comment.
(we are also in the room and received it over federation) and enters the current state (sync's
always_include_ids), while the copy being filtered (e.g. from a worker's not-yet-invalidated cache) still carries the outlier flag.
Would a better fix be to fix the cache problem?
filter_and_transform_events_for_clientcomputes the requesting user's membership (MSC4115) from the state after each event, fetched only for non-outlier events.An event let through by
always_include_idstherefore has no state to look in if it is flagged as an outlier — which happens when an out-of-band membership event is later de-outliered (we are also in the room and received it over federation) and enters the current state (sync'salways_include_ids), while the copy being filtered (e.g. from a worker's not-yet-invalidated cache) still carries the outlier flag.This hit the 'unreachable' raise, failing the whole /sync with a 500. Return the event without a membership annotation instead (MSC4115 permits omitting it), and downgrade the neighbouring call-invite assert with the same reachability hole to a guard.
Fixes #19858.
Pull Request Checklist
EventStoretoEventWorkerStore.".code blocks.