arXiv:2602.08676cs.LGcs.AI2026-02被引 45

通过词元编辑提升文本扩散模型生成速度,兼顾效率与质量。

LLaDA2.1: Speeding Up Text Diffusion via Token Editing

  • 融合词元到词元编辑与掩码到词元框架,实现可配置的阈值解码。
  • 100B模型在代码任务上达892 TPS,比传统方法快数倍。
  • 支持速度与质量双模式,适合追求高效生成的开发者使用。

尽管LLaDA2.0展示了百亿级块扩散模型的扩展潜力及其固有的并行性,但解码速度与生成质量之间的微妙平衡仍是一个未解难题。本文提出LLaDA2.1,通过将词元到词元(T2T)编辑无缝融入传统的掩码到词元(M2T)方案,引入一种联合可配置的阈值解码机制。该结构创新催生两种模式:速模式(S Mode)降低M2T阈值以突破传统限制,依赖T2T优化输出;质模式(Q Mode)采用保守阈值以保障基准性能,效率略有下降。此外,在超大上下文窗口支持下,首次构建专为dLLMs设计的大规模强化学习(RL)框架,结合稳定梯度估计技术,显著提升推理精度与指令遵循能力,弥合扩散动态与复杂人类意图之间的鸿沟。本文发布LLaDA2.1-Mini(16B)和LLaDA2.1-Flash(100B)。在33项严格基准测试中,LLaDA2.1表现优异,解码速度极快:代码任务上,HumanEval+达892 TPS,BigCodeBench达801 TPS,LiveCodeBench达663 TPS。

原文摘要 · Abstract (English)

While LLaDA2.0 showcased the scaling potential of 100B-level block-diffusion models and their inherent parallelization, the delicate equilibrium between decoding speed and generation quality has remained an elusive frontier. Today, we unveil LLaDA2.1, a paradigm shift designed to transcend this trade-off. By seamlessly weaving Token-to-Token (T2T) editing into the conventional Mask-to-Token (M2T) scheme, we introduce a joint, configurable threshold-decoding scheme. This structural innovation gives rise to two distinct personas: the Speedy Mode (S Mode), which audaciously lowers the M2T threshold to bypass traditional constraints while relying on T2T to refine the output; and the Quality Mode (Q Mode), which leans into conservative thresholds to secure superior benchmark performances with manageable efficiency degrade. Furthering this evolution, underpinned by an expansive context window, we implement the first large-scale Reinforcement Learning (RL) framework specifically tailored for dLLMs, anchored by specialized techniques for stable gradient estimation. This alignment not only sharpens reasoning precision but also elevates instruction-following fidelity, bridging the chasm between diffusion dynamics and complex human intent. We culminate this work by releasing LLaDA2.1-Mini (16B) and LLaDA2.1-Flash (100B). Across 33 rigorous benchmarks, LLaDA2.1 delivers strong task performance and lightning-fast decoding speed. Despite its 100B volume, on coding tasks it attains an astounding 892 TPS on HumanEval+, 801 TPS on BigCodeBench, and 663 TPS on LiveCodeBench.

文本生成扩散模型加速推理强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。