通过单序列树状注意力并行解码，让推理更快
Accelerate Parallelizable Reasoning via Parallel Decoding within One Sequence
- 用树状注意力在单序列中并行生成多个推理步骤
- 解码速度最高提升近100%，答案质量基本不变
- 适合需要快速推理但又不牺牲准确性的场景
近期的推理模型通过详细且全面的推理过程显著提升了准确性。然而，生成这些长推理序列计算成本高、耗时长。为解决此效率问题，我们利用某些任务固有的可并行性，加速推理过程。具体而言，当存在多个可并行的推理步骤时，我们在单个序列中通过树状注意力掩码实现每前向传播一次解码多个标记，避免额外内存开销。实验结果表明，该方法在保持答案质量基本不变的前提下，解码速度最高可达近100%的提升。
原文摘要 · Abstract (English)
Recent advances in reasoning models have demonstrated significant improvements in accuracy by employing detailed and comprehensive reasoning processes. However, generating these lengthy reasoning sequences is computationally expensive and time-consuming. To address this inefficiency, we leverage the inherent parallelizability of certain tasks to accelerate the reasoning process. Specifically, when multiple parallel reasoning steps exist, we decode multiple tokens per forward pass via a tree-like attention mask within a single sequence, avoiding additional memory usage. Experimental results show that our method achieves up to nearly 100\% speedup in decoding while basically maintaining the answer quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。