Skip to content

fix(video): sync the video sink on resize instead of waiting for a frame#273

Open
rmounce wants to merge 2 commits into
selkies-project:comprehensive-fixesfrom
rmounce:rmounce/cf-resize-stale-frame
Open

fix(video): sync the video sink on resize instead of waiting for a frame#273
rmounce wants to merge 2 commits into
selkies-project:comprehensive-fixesfrom
rmounce:rmounce/cf-resize-stale-frame

Conversation

@rmounce

@rmounce rmounce commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

On a window resize, the canvas-style writers (applyManualCanvasStyle / resetCanvasStyle) unconditionally re-show the main canvas and rewrite its box; the active video sink (<video>/worker canvas) is only re-hidden behind it and re-mirrored to the new geometry by the present paths — i.e. on the next decoded frame. On a static remote there is no next frame, so after a resize the stale canvas content (typically the connection-time picture) covers the live sink, correctly scaled — a "freeze-frame" that tracks the window — until motion produces a frame. Reproduced in both Chromium (MSTG sink) and Firefox (worker canvas sink).

Fix: when the active sink has proven it renders, sync it directly from the style writers — mirror the fresh canvas box onto it and re-hide the canvas immediately, no frame required. Warm-up behavior (canvas visible until the sink's first render) is unchanged.

Tested on the LSIO-based image against comprehensive-fixes (da1b55e): resize with a static remote now stays on the live stream in both browsers.

🤖 Generated with Claude Code

Co-authored-by: vishalkadam47 <vishalkadam472@gmail.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces the syncSinkToCanvasStyle function to synchronize the active video sink's style with the canvas style and hide the canvas immediately once rendering is confirmed. This function is integrated into both applyManualCanvasStyle and resetCanvasStyle. The review feedback highlights a potential state mismatch in syncSinkToCanvasStyle where mstgLastGeom could be incorrectly updated instead of videoWorkerLastGeom if mstgActive is true but videoElement is null. A robust solution was suggested to explicitly track which active flag was used to select the target.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +1388 to +1406
function syncSinkToCanvasStyle() {
if (!canvas) return;
let target = null, rendered = false;
if (mstgActive && videoElement) {
target = videoElement;
rendered = mstgRendered;
} else if (videoWorkerActive) {
target = (videoWorkerMode === 'vtg') ? videoElement : videoWorkerCanvas;
rendered = videoWorkerRendered;
}
if (!target) return;
const geom = canvas.style.cssText; // capture while the canvas is visible
target.style.cssText = geom;
target.style.display = 'block';
target.style.objectFit = 'fill';
if (mstgActive) mstgLastGeom = geom; else videoWorkerLastGeom = geom;
canvasGeomDirty = false;
if (rendered) canvas.style.display = 'none';
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

In syncSinkToCanvasStyle, the target is selected using the condition mstgActive && videoElement. However, the decision of which geom variable to update (mstgLastGeom vs videoWorkerLastGeom) is determined solely by mstgActive. If mstgActive is true but videoElement is null, the target will fall back to the worker target (if videoWorkerActive is true), but mstgLastGeom will still be updated instead of videoWorkerLastGeom.\n\nTo make this logic robust and prevent potential state mismatch, we should explicitly track which active flag was used to select the target (e.g., using an isMstg boolean flag) and use that to update the corresponding geom variable.

function syncSinkToCanvasStyle() {\n  if (!canvas) return;\n  let target = null, rendered = false, isMstg = false;\n  if (mstgActive && videoElement) {\n    target = videoElement;\n    rendered = mstgRendered;\n    isMstg = true;\n  } else if (videoWorkerActive) {\n    target = (videoWorkerMode === 'vtg') ? videoElement : videoWorkerCanvas;\n    rendered = videoWorkerRendered;\n  }\n  if (!target) return;\n  const geom = canvas.style.cssText;   // capture while the canvas is visible\n  target.style.cssText = geom;\n  target.style.display = 'block';\n  target.style.objectFit = 'fill';\n  if (isMstg) mstgLastGeom = geom; else videoWorkerLastGeom = geom;\n  canvasGeomDirty = false;\n  if (rendered) canvas.style.display = 'none';\n}

The canvas-style writers (applyManualCanvasStyle / resetCanvasStyle),
which run on every window resize, unconditionally re-show the main
canvas and rewrite its box; the active video sink (<video> or worker
canvas) is only re-hidden behind and re-mirrored from it by the present
paths, i.e. on the NEXT decoded frame. On a static remote there is no
next frame, so after a resize the stale canvas content (last painted
during sink warm-up, typically the connection-time picture) covers the
live sink - correctly scaled, which makes it look like a freeze-frame
that tracks the window - until motion produces a frame. Observed in
both Chromium (MSTG sink) and Firefox (worker canvas sink).

When the active sink has proven it renders, sync it directly from the
style writers: mirror the fresh canvas box onto it and re-hide the
canvas immediately, no frame required. During warm-up (nothing rendered
on the sink yet) the canvas is left visible, as before.

Co-authored-by: Claude <noreply@anthropic.com>
@rmounce rmounce force-pushed the rmounce/cf-resize-stale-frame branch from 514ce4e to f1cb686 Compare July 8, 2026 08:01
@ehfd ehfd force-pushed the comprehensive-fixes branch 2 times, most recently from 32a3087 to 6f4e5c7 Compare July 8, 2026 09:13
@ehfd

ehfd commented Jul 8, 2026

Copy link
Copy Markdown
Member

Incorporated into #254

@ehfd ehfd force-pushed the comprehensive-fixes branch 2 times, most recently from 978c4d0 to 311d338 Compare July 8, 2026 12:18
@ehfd ehfd force-pushed the comprehensive-fixes branch 8 times, most recently from 3bab531 to 4de4c8a Compare July 9, 2026 07:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants