HeadCast

Casting Attention Heads for Efficient Autoregressive Video Generation

---

HeadCast accelerates autoregressive video diffusion by classifying attention heads into distinct behavioral groups (e.g. sink / dummy / global / spatial) and allocating a heterogeneous, per-group context (KV-cache) budget accordingly. This head-level context allocation cuts attention cost while preserving generation quality, enabling efficient high-resolution and long-context video generation.

This repository unifies the Self-Forcing and LongLive autoregressive video-generation pipelines, so you can switch between models simply by changing the config file.

Get Started

NOTE: At least 40GB GPU memory is recommended.

Installation

Create a conda environment and install dependencies (tested with CUDA 13.0 on an H100; adjust the torch version for your setup):

conda create -n headcast python=3.10 -y
conda activate headcast
pip install torch==2.9.0 torchvision==0.24.0 torchaudio==2.9.0 --index-url https://download.pytorch.org/whl/cu130
pip install -r requirements.txt
pip install flash-attn --no-build-isolation

Download the pre-trained checkpoints into ./checkpoints (the default path referenced by the configs):

huggingface-cli download Wan-AI/Wan2.1-T2V-1.3B          --local-dir ./checkpoints/Wan-AI/Wan2.1-T2V-1.3B
huggingface-cli download Efficient-Large-Model/LongLive-1.3B --local-dir ./checkpoints/LongLive-1.3B
huggingface-cli download gdhe17/Self-Forcing            --local-dir ./checkpoints/Self-Forcing

The configs already point to ./checkpoints/...; adjust the paths in configs/*.yaml and utils/wan_wrapper.py if you place the weights elsewhere.

5s Short Video Generation

# Self-Forcing model
python inference.py --config_path configs/self_forcing_inference.yaml

# LongLive model
python inference.py --config_path configs/longlive_inference.yaml

Generated videos are stored under ./videos. Edit the prompts in ./prompts/ for customization.

30s Long Video Generation

Set num_output_frames: 120 in configs/longlive_inference.yaml, then:

python inference.py --config_path configs/longlive_inference.yaml

VBench Evaluation

# self-forcing model
torchrun --nproc_per_node=1 --master_port=39500 sample_vbench.py --config_path configs/self_forcing_vbench.yaml

# longlive model
torchrun --nproc_per_node=1 --master_port=29500 sample_vbench.py --config_path configs/longlive_vbench.yaml

The command generates several videos per prompt into a single folder for evaluation. See the official VBench repo for evaluation details.

License

Released under the Apache License 2.0 (see LICENSE.txt).

Acknowledgements

Built upon Self-Forcing, LongLive, and Wan.