[OS 3.8 Regression] Remote Play Steam Deck decoder performance noticeable worse compared to 3.7

[DeNo64]

Your system information

• Steam client version: 1782533657
• SteamOS version: 3.8.11
• Opted into Steam client beta?: No
• Opted into SteamOS beta?: No
• Have you checked for updates in Settings > System?: Yes

Please describe your issue in as much detail as possible:

Updating to OS 3.8.11 (from 3.7.25) has introduced at least two issue, I assume as a result of using 'VAAPI DRM hardware decoding' (3.8) vs. 'VAAPI tiled hardware decoding' (3.7).

1. Steam Menu & Quick Access Menu are invisible (sometimes flickering on quickly. But basically unusable)
2. Decoding latency increases by ~3ms (eyeballing the graph)

All remote play settings are the same, Steam Client versions are the same, and rolling back to 3.7.25 reducing the latency back down to the original levels, and the overlay menu are visible and don't flicker.

TEST
There's no specific values I can see to quantify just the decode latency before and after. But based on the line graph I have estimated the encode vs decode times

Game streamed: Expedition 33
Host: NVIDIA powered Arch Linux (doubt anything else is relevant since encoding time don't seem to change)

Resolution: 1440p
Bitrate: Automatic
Framerate: Automatic
Encoding: HEVC
Decoding: Hardware
Low Latency: Enabled

Actual Values
3.7.25
Capture 2560x1440 @ 60.00
Encoder: Game Vulkan NV12 + NVENC HEVC
Decoder: VAAPI tiled hardware decoding

Streaming Latency: (1ms input, ~6.67ms display) - estimated (Encoding: ~5ms, Decoding: ~2ms)
Ping time: <1ms (direct)

Incoming bitrate: ~30282 kbit/s video: ~27592 kbit/s Outgoing bitrate: ~194 kbit/s

3.8.11
Capture 2560x1440 @ 60.00
Encoder: Game Vulkan NV12 + NVENC HEUC
Decoder: VAAPI DRM hardware decoding

Streaming Latency: (1ms input, ~10.38ms display) - estimated (Encoding: ~5ms, Decoding: ~5ms)
Ping time: <1ms (direct)

Incoming bitrate: ~31909 kbit/s video: ~29096 kbit/s
Outgoing bitrate: ~187 kbit/s

I have done similar tests with;

• Lego Star Wars: Skywalker Saga (7.8-8ms -> 10ms)
• Spider-Man Remastered (7.2-8.2ms -> 10.8-11.2ms)
• Battlefield 6 (dual-boots to Windows). (14-14.5ms -> 17-17.5ms) Encode: (~11ms), Decode: (3-3.5ms -> ~6ms)
  Encoder: Game D3D11 shared NV12 + NVENC HEVC

Based on the graph the increase is only coming from decoding times. All result in the same ~3ms increase to decoding latency.

Also an additional note, not sure if this is expected due to OLED having the higher clocked ram or consistent CPU clock, but LCD Steam Deck on OS 3.8.11 seems to add an additional ~2ms decoding (based on eyeballing the graph) compared to OLED on 3.8.11. Both using the DRM decoder.

Steps for reproducing this issue:

1. On OS 3.7.25 launch game in remote play (Streaming display latency ~6.5 - 7ms)
2. Update OS to 3.8.11
3. Launch game in remote play
   (Streaming display latency ~10.5ms