用强化学习动态指导推理路径，让大模型更高效地解决复杂问题。
Policy Guided Tree Search for Enhanced LLM Reasoning
- 用学习到的策略自动决定探索方向，替代人工设计规则。
- 在数学与逻辑任务上表现优于现有方法，计算成本降低显著。
- 适合需要深度推理的场景，如数学证明、复杂规划任务。
尽管大语言模型具备强大能力，但在需要复杂推理与规划的任务上仍表现不佳。现有方法如思维链提示和树搜索虽有潜力，却受限于预设启发式规则和高计算开销的探索策略。我们提出策略引导树搜索（PGTS），结合强化学习与结构化树状探索，高效导航推理路径。核心创新在于学习一个动态策略，可自动决策是否扩展、分叉、回溯或终止探索，无需手动设计启发式规则或进行穷尽搜索。在数学推理、逻辑推断及规划基准测试中，PGTS展现出更优的推理性能，同时显著降低计算成本。结果表明，PGTS是一种可扩展且高效的复杂推理解决方案。
原文摘要 · Abstract (English)
Despite their remarkable capabilities, large language models often struggle with tasks requiring complex reasoning and planning. While existing approaches like Chain-of-Thought prompting and tree search techniques show promise, they are limited by their reliance on predefined heuristics and computationally expensive exploration strategies. We propose Policy-Guided Tree Search (PGTS), a framework that combines reinforcement learning with structured tree exploration to efficiently navigate reasoning paths. Our key innovation is a learned policy that dynamically decides between expanding, branching, backtracking, or terminating exploration, eliminating the need for manual heuristics or exhaustive search. Experiments across mathematical reasoning, logical deduction, and planning benchmarks demonstrate that PGTS achieves superior reasoning performance while significantly reducing computational costs compared to existing methods. These results establish PGTS as a scalable and effective solution for tackling complex reasoning tasks with LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。