提出LSP调度器,让扩散语言模型生成更快速连贯。
Beyond Scattered Acceptance: Fast and Coherent Inference for DLMs via Longest Stable Prefixes
- 用连续前缀一次性提交高置信度词元,避免碎片化更新。
- 在多个任务上提速最高3.4倍,且保持或小幅提升生成质量。
- 无需训练、适配任意模型,适合追求高效推理的研究与应用。
扩散语言模型(DLMs)理论上支持高度并行的文本生成,但实际推理速度常受制于低效的解码调度。传统方法依赖‘分散接受’——在序列不同位置逐个提交高置信度词元,导致键值缓存(KV cache)碎片化、破坏内存局部性,并在不稳定词元边界引发频繁修复。为此,我们提出最长稳定前缀(LSP)调度器,一种无需训练、模型无关的推理范式,基于整体前缀吸收机制。每轮去噪过程中,LSP通过单次前向传播评估词元稳定性,动态识别一个连续的左对齐稳定预测块,并在自然语言或结构分隔符处原子化提交其边界。该前缀优先结构带来双重优势:系统层面,将碎片化缓存更新转为高效连续追加;算法层面,保留几何级缩小的活跃后缀双向前瞻能力,显著降低词元翻转率与去噪器调用次数。在LLaDA-8B和Dream-7B上的广泛评估表明,LSP在数学推理、代码生成、多语言(中日韩)任务及创意写作等严格基准上,推理速度最高提升3.4倍,同时匹配或轻微改善输出质量。通过根本重构提交拓扑,LSP弥合了DLM理论并行性与实际硬件效率之间的鸿沟。
原文摘要 · Abstract (English)
Diffusion Language Models (DLMs) promise highly parallel text generation, yet their practical inference speed is often bottlenecked by suboptimal decoding schedulers. Standard approaches rely on 'scattered acceptance'-committing high confidence tokens at disjoint positions throughout the sequence. This approach inadvertently fractures the Key-Value (KV) cache, destroys memory locality, and forces the model into costly, repeated repairs across unstable token boundaries. To resolve this, we present the Longest Stable Prefix (LSP) scheduler, a training-free and model-agnostic inference paradigm based on monolithic prefix absorption. In each denoising step, LSP evaluates token stability via a single forward pass, dynamically identifies a contiguous left-aligned block of stable predictions, and snaps its boundary to natural linguistic or structural delimiters before an atomic commitment. This prefix-first topology yields dual benefits: systemically, it converts fragmented KV cache updates into efficient, contiguous appends; algorithmically, it preserves bidirectional lookahead over a geometrically shrinking active suffix, drastically reducing token flip rates and denoiser calls. Extensive evaluations on LLaDA-8B and Dream-7B demonstrate that LSP accelerates inference by up to 3.4x across rigorous benchmarks including mathematical reasoning, code generation, multilingual (CJK) tasks, and creative writing while matching or slightly improving output quality. By fundamentally restructuring the commitment topology, LSP bridges the gap between the theoretical parallelism of DLMs and practical hardware efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。