Skip to content

Feat: Temporal tile custom size with overlap#1510

Open
stduhpf wants to merge 1 commit into
leejet:masterfrom
stduhpf:temporal-tiling-improvements
Open

Feat: Temporal tile custom size with overlap#1510
stduhpf wants to merge 1 commit into
leejet:masterfrom
stduhpf:temporal-tiling-improvements

Conversation

@stduhpf
Copy link
Copy Markdown
Contributor

@stduhpf stduhpf commented May 17, 2026

Summary

This PR improves the temporal tiled decoding for the LTX2.3 Video VAE by adding support for custom tile sizes (processing multiple latent frames per batch) and temporal overlap/padding to ensure smoother transitions between "tiles" as the VAE decoder has access to the near future.

For now it ses env variables VAE_TILE_FRAMES and VAE_TILE_PAD to control the effect. In my experience, setting VAE_TILE_FRAMES=4 and VAE_TILE_PAD=1 seems to give very decent results (see comment).

Related Issue / Discussion

#1490

Additional Information

How it works:

Currently, when temporal tiling is enabled, it switches from decoding all latent frames at once to decoding them sequentially one-by-one. However, this causes choppy transitions every 8 frames (corresponding to 1 latent frame).

To fix this, instead of processing one frame at a time, latent frames are now decoded in small, overlapping batches (tiles). Each batch has access to the cached context of all the previous ones (like it is already the case when decoding frames sequentially). To ensure seamless transitions, the final frame(s) of a batch act as an overlap; they are discarded from the current output and re-processed at the start of the next batch.

Quick comparison

No tiling frame-by frame tile = 4 , pad = 1
result
output.-.full.webm
output.-.tiling.webm
output.-.t4p1.webm
compute buffer size 14381.63 MB 2933.75 MB 10143.13 MB
decode_first_stage time 28.77s 13.64s 18.09s
tiles processed 1 (or 0?) 13 5

Checklist

@stduhpf stduhpf changed the title Feat: Temporal tile custom sieze with overlap Feat: Temporal tile custom size with overlap May 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant