arXiv:2605.29233cs.LGcs.AI2026-05

通过多尺度分块并行,让扩散语言模型更快生成文本。

BlockBatch: Multi-Scale Consensus Decoding for Efficient Diffusion Language Model Inference

论文配图:BlockBatch: Multi-Scale Consensus Decoding for Efficient Diffusion Language Model Inference
图 1 · 摘自论文原文
  • 同一请求用多种分块大小并行计算,共享初始信息,分叉处理关键语义点。
  • 平均减少26.6%去噪步数,端到端速度提升1.33倍,准确率不变。
  • 无需训练,适合追求推理加速的扩散语言模型应用者。

扩散语言模型(dLLMs)通过并行去噪多个词元位置生成文本,是自回归解码的有力替代方案。然而,块式推理面临粒度权衡:小块保留局部条件性但需更多去噪步骤,大块提升并行性却易过早决策并累积缓存误差。现有加速方法通常对每请求固定块大小,未能利用不同块大小间的互补性。本文发现块大小本身是一个有用的分支维度:不同块大小引发相关但非相同的键值缓存轨迹——各分支常共享初始前缀,在语义关键点分叉,后又在语法轻量词上达成一致。受此启发,提出BlockBatch:一种无需训练的在线推理框架,将同一请求的多块大小分支在批量前向中协同执行。通过置信度门控合并、基于领导者的同步和周期性全序列刷新机制,协调分支间一致性,重锚局部更新至全局一致的键值状态。在3个代表性dLLM和4个数据集上,BlockBatch平均减少26.6%的去噪有效步数(NFE),相比Fast-dLLM实现1.33倍平均端到端加速,同时保持精度。结果表明,块大小多样性是分支并行推理中一个实用且长期被忽视的优化轴。

原文摘要 · Abstract (English)

Diffusion language models (dLLMs) generate text by iteratively denoising multiple token positions in parallel, offering an attractive alternative to strictly autoregressive decoding. In practice, however, block-wise dLLM inference exposes a difficult granularity trade-off: small blocks preserve local conditioning but require many denoising steps, whereas large blocks expose more parallelism but can make premature commitments and accumulate cache error. Existing acceleration methods typically choose a single block size per request, leaving the complementarity among block sizes unused. We show that block size itself is a useful branching dimension. Different block sizes induce related but non-identical KV-cache trajectories: branches often share an initial prefix, bifurcate at semantically decisive positions, and later agree on syntactically lightweight tokens. Motivated by this structure, we propose BlockBatch, a training-free online inference framework that executes multiple block-size branches for the same request inside a batched forward pass. BlockBatch coordinates these branches through confidence-gated token merging, leader-based synchronization, and periodic full-sequence refreshes that re-anchor local block updates to a globally consistent KV state. Across 3 representative dLLMs and 4 datasets, BlockBatch reduces denoising NFEs by 26.6\% on average and achieves a 1.33$\times$ average end-to-end speedup over Fast-dLLM while preserving accuracy. These results identify block-size diversity as a practical and previously underexplored axis for branch-parallel dLLM inference.

扩散模型推理加速并行生成KV缓存

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。