Skip to content

Vulkan: fix block-compressed texture upload#11

Open
Monstrofil wants to merge 1 commit into
mainfrom
fix/vulkan-bc-texture-upload
Open

Vulkan: fix block-compressed texture upload#11
Monstrofil wants to merge 1 commit into
mainfrom
fix/vulkan-bc-texture-upload

Conversation

@Monstrofil

Copy link
Copy Markdown

Summary

  • MappedTexture::Map / Unmap sized the staging buffer and ImagePlane pitch with width * height * BytesPerPixel. The format mapping table lists BytesPerPixel = 0 for DXT/BC formats, so the multiplication gave a 0-byte staging buffer and Map() also bailed at the BytesPerPixel == 0 gate. Every BC-format upload was silently dropped -- the VkImage stayed at VK_IMAGE_LAYOUT_UNDEFINED and sampled as black.
  • Switch the four size/pitch sites in Vulkan_Texture.cpp to ImageData::GetMipLevelSize / GetFormatPitch, which already do the right block math for BC/ATC and pixel math for everything else. Replace the BytesPerPixel == 0 early-exit with a totalSize == 0 guard.
  • D3D11 doesn't have this issue because it uses USAGE_STAGING textures and trusts D3D11_MAPPED_TEXTURE2D::RowPitch from the driver. Vulkan uses staging buffers + vkCmdCopyBufferToImage, where buffer layout is the application's responsibility.

Reproducer

Load any DXT5 .dds via flash.display.Loader.load on the Vulkan backend (e.g. WoWP loading.swf's gui/flash/login/backgrounds/logoLogIn_*.dds). Pre-fix: black background. Post-fix: image renders.

Test plan

  • WoWP loading.swf background DDS now renders correctly under the Vulkan backend.
  • Run the GFx Vulkan sample suite to confirm no regressions on uncompressed paths.

MappedTexture::Map and Unmap sized the staging buffer and ImagePlane
pitch with (width * height * BytesPerPixel). For block-compressed
formats (DXT/BC) the mapping table sets BytesPerPixel = 0, so the
multiplication produced a 0-byte staging buffer and Map() also bailed
out at the BytesPerPixel == 0 gate. Net effect: every BC upload was
silently dropped, the VkImage stayed at VK_IMAGE_LAYOUT_UNDEFINED,
and the bitmap sampled as black.

Add VkUploadPitch / VkUploadLevelSize helpers that branch on the
mapping's BytesPerPixel:

  - BytesPerPixel != 0 (uncompressed): keep the original
    width * BytesPerPixel math. This is the *destination* (Vulkan)
    bytes per pixel, which matters when a format conversion happens
    in the scanline copy -- e.g. Image_R8G8B8 maps to
    VK_FORMAT_R8G8B8A8_UNORM via Image_CopyScanline24_Extend_RGB_RGBA,
    so the staging buffer holds 4 bpp even though the source is 3 bpp.
    Switching this site to ImageData::GetFormatPitch (which returns
    *source* pitch) undersizes the staging buffer for these
    expanding-conversion formats and the scanline copy runs off the
    end -- crash on embedded JPEG decode.

  - BytesPerPixel == 0 (compressed): fall back to
    ImageData::GetMipLevelSize / GetFormatPitch, which already do the
    correct 4x4 block math for BC/ATC and don't need any conversion
    (BC scanlines are copied verbatim).

Replace the BytesPerPixel == 0 early-exit in Map() with a
totalSize == 0 guard so BC formats now pass through.

D3D11 doesn't have this issue because it uses USAGE_STAGING textures
and trusts D3D11_MAPPED_TEXTURE2D::RowPitch from the driver; Vulkan
uses staging *buffers* + vkCmdCopyBufferToImage where buffer layout
is the application's responsibility.

Reproducer: load any DXT5 .dds via flash.display.Loader.load on the
Vulkan backend (e.g. WoWP loading.swf's logoLogIn_*.dds backgrounds).
Pre-fix: black background. Post-fix: image renders. Verified: SWFs
with embedded 24-bit JPEGs (e.g. login.swf) still load without
regression.
@Monstrofil Monstrofil force-pushed the fix/vulkan-bc-texture-upload branch from 0956c49 to 027d963 Compare May 3, 2026 19:10
@Monstrofil

Copy link
Copy Markdown
Author

Updated the patch after a regression: my first version replaced width * BytesPerPixel with ImageData::GetFormatPitch everywhere. That broke uncompressed formats with format conversion (e.g. Image_R8G8B8 -> VK_FORMAT_R8G8B8A8_UNORM via Image_CopyScanline24_Extend_RGB_RGBA), since GetFormatPitch returns the source (3 bpp) pitch but the staging buffer holds the destination (4 bpp). SWFs with embedded 24-bit JPEGs crashed in Image_CopyScanline24_Extend_RGB_RGBA because the scanline copy ran off the end.

The amended commit branches on the mapping's BytesPerPixel:

  • != 0 (uncompressed): keep the original width * BytesPerPixel math (destination layout, handles format-converting copies correctly).
  • == 0 (compressed): use ImageData::GetMipLevelSize / GetFormatPitch (block math; no conversion happens for BC).

Verified: WoWP loading.swf DXT5 background still renders, and login.swf (which embeds 24-bit JPEGs) no longer crashes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant