让大模型推理更快更准,无需特殊引擎
ThreadWeaver: Adaptive Threading for Efficient Parallel Reasoning in Language Models
- 自适应并行生成思维链数据,提升训练质量
- 在普通自回归引擎上实现并行推理,最高提速1.53倍
- 结合强化学习优化准确率与并行效率平衡,适合实际部署
扩展推理阶段计算使大语言模型在推理任务中表现优异,但其固有的序列解码导致延迟较高。现有并行推理方法常导致性能下降且依赖专用推理引擎。本文提出ThreadWeaver框架,通过三项创新实现高效并行推理:1)两阶段并行轨迹生成器,用于监督微调的高质量并行思维链数据;2)基于前缀树的展开设计,可在任意现成的自回归推理引擎上实现并行推理;3)面向并行化的强化学习训练框架,使模型在推理准确率与并行效率间取得平衡。在六个数学推理基准测试中,基于Qwen3-8B训练的ThreadWeaver达到与顶尖串行模型相当的性能(AIME24为79.9%,平均为71.9%),同时实现最高1.53倍的令牌延迟加速,确立了准确率与效率的新权衡前沿。
原文摘要 · Abstract (English)
Scaling inference-time computation has enabled Large Language Models (LLMs) to achieve strong reasoning performance, but their inherently sequential decoding incurs substantial latency, motivating parallelization of the generation process. However, existing parallel reasoning approaches suffer from performance degradation compared to their sequential counterparts, and often rely on specialized inference engines. We introduce ThreadWeaver, a framework for adaptive parallel reasoning that matches the accuracy of comparably sized sequential reasoning models while significantly reducing inference latency via three key innovations: 1) a two-stage parallel trajectory generator that produces high-quality parallel chain-of-thought data for supervised fine-tuning; 2) a trie-based rollout design that enables parallel reasoning on any off-the-shelf autoregressive inference engine; and 3) a parallelization-aware reinforcement learning framework that trains the model to balance reasoning accuracy with effective parallelization. Across six challenging math reasoning benchmarks, ThreadWeaver trained on top of Qwen3-8B achieves performance on par with cutting-edge sequential reasoning models (79.9% on AIME24 and 71.9% on average) while delivering up to 1.53x speedup in token latency, establishing a new Pareto frontier between accuracy and efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。