Skip to content

fix(windows-lib): release thumbnails after rendered frame - #519

Open
sim590 wants to merge 2 commits into
H3rmt:hyprshellfrom
sim590:fix-vram-leak
Open

fix(windows-lib): release thumbnails after rendered frame#519
sim590 wants to merge 2 commits into
H3rmt:hyprshellfrom
sim590:fix-vram-leak

Conversation

@sim590

@sim590 sim590 commented Aug 30, 2026

Copy link
Copy Markdown

Problem

Live window thumbnails leave a significant amount of GPU memory allocated
after the overview or switch surface is closed. Reopening the surface can
also leave stale GLib callbacks behind, causing the next invocation to fail
or crash.

Root cause

Thumbnail widgets keep references to GDK textures backed by imported DMA-BUF
images. Hiding the layer surface before GTK renders an updated scene prevents
GSK from observing that those images are no longer used. The images are then
kept by the GSK GPU cache until its normal garbage-collection timeout.

Fix

Clear the thumbnail widgets before hiding the surface and wait for the
GdkFrameClock after-paint signal. This ensures that a frame without the
thumbnail textures has been rendered before the surface is hidden.

The overview waits for all monitor windows to complete this step before
performing a pending client or workspace switch. Frame handlers are
disconnected safely when a surface is reopened.

Refresh callbacks use cancellation tokens instead of removing potentially
expired GLib sources, avoiding crashes during rapid close and reopen cycles.

DMA-BUF lifetime

Keep a duplicated file descriptor alive for each GDK DMA-BUF texture instead
of relying on a bounded list of retired GBM buffer objects. This makes the
DMA-BUF lifetime follow the texture that imported it and prevents closed file
descriptors from being reused after buffer reallocation.

This is an independent lifetime-safety fix for resize and reallocation
scenarios; it is not required for the normal VRAM cleanup path described
above.

sim590 added 2 commits August 30, 2026 18:36
Keep a duplicated FD alive for every GDK DMA-BUF texture instead of
retaining a bounded set of GBM buffer objects. Store the modifier in
WindowCapture so GBM objects can be released immediately.

This prevents GSK from using a closed FD after buffer reallocation and
makes each DMA-BUF lifetime follow the texture that imports it.
Clear thumbnail widgets before hiding the overview or switch surface,
then request an empty frame and hide the surface from its after-paint
callback. This lets GSK discard imported DMA-BUF images without a fixed
delay.

Wait for every overview window before applying a pending client or
workspace switch, and safely disconnect frame handlers during rapid
reopen cycles.

Use cancellable refresh handlers to avoid removing expired GLib sources
when the components are closed while a refresh is pending.
@sim590

sim590 commented Aug 30, 2026

Copy link
Copy Markdown
Author

@H3rmt: I noticed that VRAM was still allocated after spawning and hiding the live thumbnails.

image

I deduced that this was memory allocated in the GSK cache that needed the proper signal to get freed. You'll notice a delay before it clears. This is due to the internal cache clearing timeout which defaults to 15 seconds. You can see no delay if you set GSK_CACHE_TIMEOUT=0.

All of this seemed to happen because window.set_visible(false); is called too early. In fact, if you do the following edit to my branch:

diff --git a/crates/windows-lib/src/switch/root.rs b/crates/windows-lib/src/switch/root.rs
index 10fa474..46b3bac 100644
--- a/crates/windows-lib/src/switch/root.rs
+++ b/crates/windows-lib/src/switch/root.rs
@@ -582,6 +582,7 @@ impl SwitchRoot {
                 });
                 *self.after_paint_handler.borrow_mut() = Some(id);
                 self.window.queue_draw();
+                self.window.set_visible(false);
             } else {
                 // No frame clock available: hide directly instead of waiting.
                 self.window.set_visible(false);

You'll notice that the VRAM is not freed at all anymore. This is because the window is hidden immediately after requesting the redraw, before GTK has a chance to process and render the queued frame.

@sim590

sim590 commented Aug 30, 2026

Copy link
Copy Markdown
Author

If you prefer, I could split the two fixes in two PRs. I just felt that the DMA-BUF lifetime fix could fit in the same patch even though it's not required.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant