探索并行推理新范式，提升大模型推理鲁棒性
A Survey on Parallel Reasoning
- 通过并发多路径思考，再收敛至答案，突破传统串行推理局限
- 提出全新分类体系，涵盖非交互、交互及高效解码策略
- 适合对大模型推理可靠性与效率有要求的研究者参考
随着大语言模型能力的提升，并行推理作为一种新兴推理范式，通过在最终答案前并行探索多个思维路径，增强了推理的鲁棒性。该方法已成为克服标准串行推理脆弱性、提升实际性能的重要趋势。本文旨在综述并总结并行推理的研究进展与挑战。首先给出并行推理的形式化定义，并厘清其与链式思维（Chain-of-Thought）等概念的区别。随后，基于新的分类体系，系统组织并讨论先进方法，包括非交互式推理、交互式推理以及面向效率的解码策略。此外，探讨了其在解决复杂问题和提升大模型输出可靠性等应用场景中的表现。最后，指出并行推理的核心挑战，并提出未来研究可能的方向。希望本工作能为初学者提供清晰路线图，推动更多关于改进并行推理方法的研究。相关资源详见 https://github.com/PPPP-kaqiu/Awesome-Parallel-Reasoning。
原文摘要 · Abstract (English)
With the increasing capabilities of Large Language Models (LLMs), parallel reasoning has emerged as a new inference paradigm that enhances reasoning robustness by concurrently exploring multiple lines of thought before converging on a final answer. It has become a significant trend to explore parallel reasoning to overcome the fragility of standard sequential methods and improve practical performance. In this paper, we aim to survey and summarize the progress and challenges of parallel reasoning. We first present a formal definition of parallel reasoning and clarify its distinction from related concepts like Chain-of-Thought. Then, we organize and discuss advanced techniques based on a novel taxonomy, including non-interactive reasoning, interactive reasoning, and efficiency-focused decoding strategies. Additionally, we explore various application scenarios, such as solving complex problems and enhancing the reliability of LLM outputs.Finally, we highlight the core challenges of parallel reasoning and suggest potential directions for future research. We hope that our work can provide a useful roadmap for beginners and encourage more research on improving parallel reasoning methods. Related source can be avaliable in https://github.com/PPPP-kaqiu/Awesome-Parallel-Reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。