AB-Attention is a training-free method that speeds up video diffusion transformers by skipping most attention work while staying close to dense attention.
Method: IBCS estimates block importance by clustering queries within each query block, without permuting tokens. SIP samples blocks in proportion to those estimates and reweights them by inverse inclusion probability. The kernel gathers smaller key/value blocks into full tensor-core tiles.
Sparsity: On Wan2.1 and HunyuanVideo, the paper says it skips 89 to 92 percent of attention, where the baselines compute 1.8 to 3.6 times as much.
Results: It improves PSNR by up to 4.4 dB over the strongest baseline and reaches up to 2.23x end-to-end speedup.
Code: The authors link a GitHub repository from the paper page.
Sign in to comment.
TL;DR of the paper, generated automatically.
AB-Attention is a training-free method that speeds up video diffusion transformers by skipping most attention work while staying close to dense attention.