用更广的初始推理路径替代深度迭代，提升大模型解题效率
Is Depth All You Need? An Exploration of Iterative Reasoning in LLMs
- 通过增加初始推理路径多样性实现广度推理，无需反复迭代
- 在多个任务上表现优于传统深度迭代推理方法
- 适合追求高效推理、减少计算开销的研究者与开发者
深度迭代思维链（CoT）推理通过逐步激活预训练知识来解决复杂任务，但面临持续改进困难和停止条件难定的问题。本文探究是否能在初始推理路径中直接激活解决当前问题所需的关联知识，从而避免迭代优化。实验表明，增加初始推理路径的多样性可达到甚至超越深度迭代的效果，这一思想称为‘广度推理’。然而，现有广度推理方法如自一致性提供的多样性有限。为此，我们提出一种简单有效的方法：结合上下文探索并降低采样随机性，以增强推理广度。大量实验显示，该方法显著优于深度迭代推理。代码已开源：https://github.com/zongqianwu/breadth。
原文摘要 · Abstract (English)
Deep iterative chain-of-thought (CoT) reasoning enables LLMs to tackle complex tasks by progressively activating relevant pre-trained knowledge. However, it faces challenges in ensuring continual improvement and determining a stopping criterion. In this paper, we investigate whether the relevant knowledge that contributes directly to solving the given question can be activated from the initial reasoning path, thus circumventing the need for iterative refinement. Our experiments reveal that increasing the diversity of initial reasoning paths can achieve comparable or superior performance, a concept we term \textit{breadth reasoning}. However, existing breadth reasoning approaches, such as self-consistency, offer limited diversity. To address this limitation, we propose a simple yet effective method that enhances reasoning breadth by integrating contextual exploration with reduced sampling randomness. Extensive experiments demonstrate that our approach significantly outperforms deep iterative reasoning. Our code is provided in https://github.com/zongqianwu/breadth.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。