Skip to content

feat(video): pause the video broadcast to hidden-tab viewers#271

Closed
rmounce wants to merge 2 commits into
selkies-project:comprehensive-fixesfrom
rmounce:rmounce/cf-viewer-video-pause
Closed

feat(video): pause the video broadcast to hidden-tab viewers#271
rmounce wants to merge 2 commits into
selkies-project:comprehensive-fixesfrom
rmounce:rmounce/cf-viewer-video-pause

Conversation

@rmounce

@rmounce rmounce commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

The web client already sends STOP_VIDEO on tab-hide and START_VIDEO (+ decoder re-init + watchdog) on tab-show. For the display's owning client this stops the capture — but for shared viewers, who never get a client_display_id, STOP_VIDEO was a silent no-op: a hidden viewer tab kept receiving the full video bitrate and discarding it. With several viewers parked on background tabs, that's most of the stream's bandwidth doing nothing.

This tracks paused viewer sockets in a set and excludes them from the primary video broadcast; capture keeps running and the socket stays connected for control/cursor/audio. On START_VIDEO the viewer rejoins and falls through to the existing shared-client resync (PIPELINE_RESETTING + scheduled IDR + cursor resend), so resume is one IDR away. A resuming paused viewer bypasses the 30 s viewer START_VIDEO throttle — resume is a real state change, and the throttle would otherwise blank quick tab switches — but still arms it, so resets can't be spammed faster than pause/resume cycles.

Found while building a multi-viewer portal on top of #254's token roles. Draft pending runtime testing on our lsio-based image (multi-viewer hide/show, rapid toggling, bandwidth verification); will mark ready once validated.

🤖 Generated with Claude Code

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a mechanism to pause video feeds for shared viewers when they hide their tab (sending STOP_VIDEO), while keeping their control, cursor, and audio connections active. This is achieved by tracking paused viewers in a new self.video_paused_viewers set and excluding them from the primary video broadcast. The reviewer identified a critical bug where non-viewer clients (such as collaborative controllers) who send STOP_VIDEO will be added to self.video_paused_viewers but never removed upon sending START_VIDEO because the discard logic is nested inside a viewer-only check. This would permanently pause their video feed. A code suggestion was provided to discard the websocket from self.video_paused_viewers regardless of the client's role.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread src/selkies/selkies.py
Comment on lines 2964 to +2982
perms = client_permissions.get(websocket)
if perms and perms.get("role") == "viewer":
now = time.time()
last_req_time = self.last_start_video_request_times.get(websocket, 0)
if now - last_req_time < 30.0:
data_logger.warning(f"Throttled START_VIDEO request from viewer {remote_address}. Ignoring.")
continue
self.last_start_video_request_times[websocket] = now
if websocket in self.video_paused_viewers:
# A paused viewer resuming (tab visible again) is a
# real state change, not spam: it must rejoin the
# broadcast set and get a decode entry point, so it
# bypasses the redundant-request throttle below —
# but still arms it, so a resume can't be used to
# spam resets any faster than pause/resume cycles.
self.video_paused_viewers.discard(websocket)
data_logger.info(f"START_VIDEO from paused viewer ({remote_address}): resuming its video feed.")
self.last_start_video_request_times[websocket] = now
else:
last_req_time = self.last_start_video_request_times.get(websocket, 0)
if now - last_req_time < 30.0:
data_logger.warning(f"Throttled START_VIDEO request from viewer {remote_address}. Ignoring.")
continue
self.last_start_video_request_times[websocket] = now

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Currently, self.video_paused_viewers.discard(websocket) is only called if the client's role is "viewer". However, non-viewer clients (such as collaborative controllers) can also be added to self.video_paused_viewers if they send a STOP_VIDEO message (e.g., when backgrounding their tab). If they subsequently send a START_VIDEO message, they will never be removed from self.video_paused_viewers, causing their video feed to be permanently paused.

We should check if the client was paused and discard them from self.video_paused_viewers regardless of their role, while still preserving the throttle-bypass logic for viewers.

Suggested change
perms = client_permissions.get(websocket)
if perms and perms.get("role") == "viewer":
now = time.time()
last_req_time = self.last_start_video_request_times.get(websocket, 0)
if now - last_req_time < 30.0:
data_logger.warning(f"Throttled START_VIDEO request from viewer {remote_address}. Ignoring.")
continue
self.last_start_video_request_times[websocket] = now
if websocket in self.video_paused_viewers:
# A paused viewer resuming (tab visible again) is a
# real state change, not spam: it must rejoin the
# broadcast set and get a decode entry point, so it
# bypasses the redundant-request throttle below —
# but still arms it, so a resume can't be used to
# spam resets any faster than pause/resume cycles.
self.video_paused_viewers.discard(websocket)
data_logger.info(f"START_VIDEO from paused viewer ({remote_address}): resuming its video feed.")
self.last_start_video_request_times[websocket] = now
else:
last_req_time = self.last_start_video_request_times.get(websocket, 0)
if now - last_req_time < 30.0:
data_logger.warning(f"Throttled START_VIDEO request from viewer {remote_address}. Ignoring.")
continue
self.last_start_video_request_times[websocket] = now
was_paused = websocket in self.video_paused_viewers
self.video_paused_viewers.discard(websocket)
perms = client_permissions.get(websocket)
if perms and perms.get('role') == 'viewer':
now = time.time()
if was_paused:
# A paused viewer resuming (tab visible again) is a
# real state change, not spam: it must rejoin the
# broadcast set and get a decode entry point, so it
# bypasses the redundant-request throttle below —
# but still arms it, so a resume can't be used to
# spam resets any faster than pause/resume cycles.
data_logger.info(f'START_VIDEO from paused viewer ({remote_address}): resuming its video feed.')
self.last_start_video_request_times[websocket] = now
else:
last_req_time = self.last_start_video_request_times.get(websocket, 0)
if now - last_req_time < 30.0:
data_logger.warning(f'Throttled START_VIDEO request from viewer {remote_address}. Ignoring.')
continue
self.last_start_video_request_times[websocket] = now

Co-authored-by: vishalkadam47 <vishalkadam472@gmail.com>
@ehfd ehfd force-pushed the comprehensive-fixes branch 2 times, most recently from d0fa2ee to da1b55e Compare July 8, 2026 06:10
@ehfd

ehfd commented Jul 8, 2026

Copy link
Copy Markdown
Member

Integrated into #254 with additional nit fixes.

The web client sends STOP_VIDEO when its tab is hidden and START_VIDEO
(with a decoder re-init and delivery watchdog) when it becomes visible
again — but only in non-shared mode. Shared viewers were doubly
excluded: the client's visibilitychange handler bypassed them ("stream
control bypassed"), and server-side STOP_VIDEO required a
client_display_id, which viewers never get — so a hidden viewer tab
kept receiving the full video bitrate and discarding it. With several
viewers parked on background tabs, that is most of the stream's
bandwidth doing nothing.

Server: track paused viewer sockets in a set and exclude them from the
primary video broadcast. The capture keeps running, and the socket
stays connected for control, cursor, and audio. On START_VIDEO the
viewer is re-added and falls through to the existing shared-client
resync (PIPELINE_RESETTING + scheduled IDR + cursor resend) — the same
machinery that lets fresh viewers join mid-stream, so resume is one IDR
away. A resuming paused viewer bypasses the 30-second viewer
START_VIDEO throttle (resume is a real state change, and the throttle
would otherwise blank quick tab switches); it still arms the throttle,
so resets can't be spammed faster than pause/resume cycles. The set is
cleaned up on socket disconnect and server shutdown.

Client: shared viewers now report tab visibility (STOP_VIDEO on hide,
START_VIDEO on show) instead of bypassing stream control. Against
servers without the pause feature this is behavior-compatible: viewer
STOP_VIDEO was already ignored, and START_VIDEO stays behind the
existing viewer throttle.

Co-authored-by: Claude <noreply@anthropic.com>
@rmounce rmounce force-pushed the rmounce/cf-viewer-video-pause branch from 828deb3 to 312bd18 Compare July 8, 2026 06:30
@rmounce

rmounce commented Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

Validated the incorporated version on comprehensive-fixes (da1b55e): backgrounding a viewer tab now pauses its video feed and foregrounding resumes it cleanly with the reset+IDR resync. The incorporation is a strict superset of this draft — among other hardening, the role-agnostic unpause also addresses the review bot's finding here, and adding STOP_VIDEO to allowed_viewer_prefixes fixes a gap this draft had (the viewer message gate silently denied the pause request). Closing as superseded. Thanks!

🤖 Generated with Claude Code

@rmounce rmounce closed this Jul 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants