The paper argues that diffusion transformers train faster partly by reorganizing their AdaLN-Zero residual gates, and it proposes a teacher-free objective built on that idea.
Finding: REPA and dispersive regularization substantially reshape the gate α, which the authors suggest is at least part of why acceleration happens.
Metric: A normalized participation ratio (nPR) tracks gate organization, and broader participation goes with better generation quality.
Method: Feature-norm regularization promotes broader gate participation, with no teacher and no pairwise comparisons.
Result: On ImageNet 256×256 with SiT-B/2, it improves FID over dispersive loss.
Sign in to comment.
TL;DR of the paper, generated automatically.
The paper argues that diffusion transformers train faster partly by reorganizing their AdaLN-Zero residual gates, and it proposes a teacher-free objective built on that idea.