arXiv:2601.03606cs.LGcs.AI2026-01被引 1

用语言模型概率指导思维树搜索,减少计算量同时保持高准确率。

Policy-Guided Search on Tree-of-Thoughts for Efficient Problem Solving with Bounded Language Model Queries

  • 用语言模型输出的概率做启发式指引,优化思维树搜索路径。
  • 在有限调用次数下,比基线方法更准或相当,三领域四模型均有效。
  • 适合对延迟和资源敏感的应用,如实时推理系统。

近期研究将状态空间搜索算法与语言模型(LM)结合,通过语言模型生成的「思维树」(ToT)进行前瞻规划,以提升问题求解性能。然而,现有搜索算法常忽略语言模型推理带来的显著计算开销,尤其在计算预算受限时。为此,本文提出利用语言模型对各思维路径赋予的概率作为启发式信息,指导思维树内的搜索过程,从而减少需评估的思维数量。基于此,我们将启发式搜索算法莱文树搜索(Levin Tree Search, LTS)适配至 ToT 框架,利用语言模型作为策略来高效探索树结构。理论分析表明,对于剪枝后的思维树,LTS 可保证展开状态数和生成思维数的上界;同时分析了温度参数对这一边界的影响。在固定语言模型调用预算下,实验显示 LTS 在三个领域(Blocksworld、PrOntoQA、Array Sorting)和四种不同语言模型上,均能持续达到与基线相当或更高的准确率。结果表明,该方法在保障效率的同时实现高性能,特别适用于延迟敏感和资源受限的场景。

原文摘要 · Abstract (English)

Recent studies explored integrating state-space search algorithms with Language Models (LM) to perform look-ahead on the token generation process, the ''Tree-of-Thoughts'' (ToT), generated by LMs, thereby improving performance on problem-solving tasks. However, the affiliated search algorithms often overlook the significant computational costs associated with LM inference, particularly in scenarios with constrained computational budgets. Consequently, we address the problem of improving LM performance on problem-solving tasks under limited computational budgets. We demonstrate how the probabilities assigned to thoughts by LMs can serve as a heuristic to guide search within the ToT framework, thereby reducing the number of thought evaluations. Building on this insight, we adapt a heuristic search algorithm, Levin Tree Search (LTS), to the ToT framework, which leverages LMs as policies to guide the tree exploration efficiently. We extend the theoretical results of LTS by showing that, for ToT (a pruned tree), LTS guarantees a bound on the number of states expanded, and consequently, on the number of thoughts generated. Additionally, we analyze the sensitivity of this bound to the temperature values commonly used in the final softmax layer of the LM. Empirical evaluation under a fixed LM query budget demonstrates that LTS consistently achieves comparable or higher accuracy than baseline search algorithms within the ToT framework, across three domains (Blocksworld, PrOntoQA, Array Sorting) and four distinct LMs. These findings highlight the efficacy of LTS on ToT, particularly in enabling cost-effective and time-efficient problem-solving, making it well-suited for latency-critical and resource-constrained applications.

思维树搜索算法语言模型效率优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。