NVIDIA Research has posted DiLaDiff, a masked diffusion language model variant that generates a continuous latent before the tokens. The page lists it as a NeurIPS 2026 publication.
The problem: The page says diffusion language models fail to capture correlations between decoded tokens, which forces a trade-off between sampling quality and throughput.
The design: Three parts: an auto-encoder fine-tuned from an existing masked diffusion model to build a semantic latent space, a latent diffusion model that learns the prior over that latent, and a consistency model that distills the prior into a few-step generator.
The claimed results: Even without distillation, the latent-guided model outperforms the masked diffusion baseline while significantly speeding up inference. With consistency distillation, the latent is generated in negligible time compared to discrete decoding.
Sign in to comment.
TL;DR of the article, generated automatically.
NVIDIA Research has posted DiLaDiff, a masked diffusion language model variant that generates a continuous latent before the tokens. The page lists it as a NeurIPS 2026 publication.