Unlocking Lossless Speedups in LLMs
via Discrete Diffusion

Subham Sekhar Sahoo†,1, Lingjie Chen†,1,2, Khiem Pham†,1,3, Jonathan Geuter†,1,4,
Junlin Chen1,5, Chaitanya Dwivedi1, Varad Pimpalkhute1, Yash Akhauri1, Alexander Moreno1, Mikhail Yurochkin1, Zhenting Wang1, Mostafa Elhoushi6, Nolan Dey6, Shane Bergsma6, Joel Hestness6, Hongyi Wang1,5, John Thickstun3, Eric Xing1, Zhengzhong Liu1
1Institute of Foundation Models    2University of Illinois Urbana-Champaign     3Cornell Tech
4Harvard University     5Rutgers University     6Cerebras Systems
†Core contributors
arXiv Code

🤗

 HuggingFace
MY ALT TEXT

(Top) Training overview for diffusion-augmented LLMs. Gray cells indicate AR-weight training, while the blue cell indicates diffusion-weight training. (Bottom Left) System throughput of Uno, the base AR model, and the baselines; refer the paper for details. (Bottom Right) Performance across agentic and long-context reasoning benchmarks.

A descriptive text for the GIF

Inference in Uno.

Key Innovations

  1. We introduce diffusion-augmented LLMs, a new class of models that features two sets of weights: AR weights, trained using the standard NTP objective responsible for quality, and lightweight diffusion weights, trained to generate multiple tokens simultaneously.
  2. Next, we propose \(\Psi\)-Spec samplers to generate tokens in parallel from the AR distribution defined by the AR weights. This delivers lossless speedups.
  3. Our method Uno, achieves higher throughput than leading speculative-decoding methods (DFlash and Eagle3) at every evaluated batch size without requiring an additional draft model and delivers up to \(3\times\) speedups over the base AR model, including at the largest batch size supported by the device.
  4. Our 8B Uno model outperforms the leading open d-LLM, the 26B DiffusionGemma, and the proprietary Mercury 2 across all evaluated benchmarks in agentic-tool use, coding, and long-context reasoning.

Uno (Ours) vs Speculative Decoding Methods

Comparison of Uno with speculative decoding methods

(a) Additional parameters introduced by each method (in millions) and peak GPU memory usage. Our method, Uno, introduces the least amount of parameters and has the lowest memory usage during inference. (b) System versus per-request throughput across batch sizes (concurrency C). Uno Pareto-dominates speculative decoding and achieves up to 2.5× speedup over the base AR model. Parentheses indicate the sampler configuration yielding the highest throughput for Uno.

BibTeX

@misc{sahoo2026unlockinglosslessspeedupsllms,
  title={Unlocking Lossless Speedups in LLMs via Discrete Diffusion}, 
  author={Subham Sekhar Sahoo and Lingjie Chen and Khiem Pham and Jonathan Geuter and Chaitanya Dwivedi and Varad Pimpalkhute and Yash Akhauri and Alexander Moreno and Mikhail Yurochkin and Zhenting Wang and Mostafa Elhoushi and Nolan Dey and Shane Bergsma and Joel Hestness and John Thickstun and Eric Xing and Zhengzhong Liu},
  year={2026},
  eprint={2609.04010},
  archivePrefix={arXiv},
  primaryClass={cs.LG},
  url={https://arxiv.org/abs/2609.04010}, 
}