Skip to content

GFxPlayerTinyVulkan: skip HAL Prepare/RestoreAfterReset on resize#10

Open
Monstrofil wants to merge 1 commit into
mainfrom
fix/vulkan-resize-nvidia-fillshape
Open

GFxPlayerTinyVulkan: skip HAL Prepare/RestoreAfterReset on resize#10
Monstrofil wants to merge 1 commit into
mainfrom
fix/vulkan-resize-nvidia-fillshape

Conversation

@Monstrofil

Copy link
Copy Markdown

Summary

On NVIDIA drivers, calling pHAL->PrepareForReset() / pHAL->RestoreAfterReset() around a swapchain rebuild leaves the Vulkan HAL in a stale state that silently drops subsequent filled triangle draws — only outline (line) draws survive. AMD drivers tolerate the stale state; NVIDIA does not, so the bug only manifests on NVIDIA.

Symptom on NVIDIA before the fix: SWF appears as wireframe outlines floating over the clear color (e.g. for Window.swf you get the rounded-rect window outline + the green REPEAT button, but the entire interior fill is missing).

Fix

Skip the reset notify cycle entirely on a swapchain resize. Just vkDeviceWaitIdle, recreate swapchain + depth + framebuffers, re-issue pMovie->SetViewport(...), continue. The HAL doesn''t cache framebuffers internally — SetMainRenderTarget is called per frame and picks up the new framebuffer naturally.

Verification

Pixel-perfect against the existing visual-regression baseline:

python Tests/VisualRegression/compare_images.py \
    Tests/VisualRegression/Reference/vulkan/Window_AS2/frame_0010.png \
    <captured frame from patched TinyVulkan> --threshold 8
{pass: true, rmse: 0.09, maxDelta: 5, mismatchCount: 0}

Tested on NVIDIA RTX 3060, Vulkan SDK 1.4.341, Windows 11, MSVC 2022.

Note: the underlying bug almost certainly lives in Render::Vulkan::HAL::RestoreAfterReset (the notify-handlers chain doesn''t restore something NVIDIA strictly requires), but pinning that down requires a RenderDoc capture across the reset boundary. This PR is the workaround that unblocks the sample on NVIDIA; a proper fix to RestoreAfterReset itself can land separately.

Test plan

  • Build TinyVulkan in Release on NVIDIA
  • Run GFxPlayerTinyVulkan.exe Bin/Data/AS2/Samples/Window.swf — verify the rounded-rect window frame fills properly (not just outlines), and the green REPEAT button is solid
  • Resize the window — frame should re-render correctly without losing fills
  • Run the Vulkan visual regression suite to confirm no regression on AMD

🤖 Generated with Claude Code

On NVIDIA drivers, calling pHAL->PrepareForReset() and pHAL->RestoreAfterReset()
around a swapchain rebuild leaves the Vulkan HAL with stale internal state, so
subsequent filled triangle draws are silently dropped. Only outline (line) draws
survive — the SWF appears as wireframe over the clear color. AMD drivers tolerate
the stale state and render correctly, which is why this went unnoticed on AMD.

The HAL doesn't cache framebuffers internally — SetMainRenderTarget is called
per frame anyway and picks up the new framebuffer naturally. So the cleanest fix
is to skip the reset notify cycle entirely on a swapchain resize: just
vkDeviceWaitIdle, recreate swapchain + depth + framebuffers, re-issue
pMovie->SetViewport, and continue.

Verified pixel-perfect against the existing visual-regression baseline (RMSE 0.09
vs Tests/VisualRegression/Reference/vulkan/Window_AS2/frame_0010.png).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant