arXiv:2605.31492cs.AI2026-05

给大模型的推理过程加结构化树形记录,提升解题效率和准确率。

LinTree: Improving LLM Reasoning with Explicitly Structured Search Histories

  • 用显式父指针构建线性搜索树,让模型清楚知道回溯到哪一步。
  • 在积木世界、网格导航等任务中,性能比传统启发式搜索提升15%以上。
  • 适合研究大模型推理机制或想优化复杂问题求解的开发者。

大型语言模型(LLMs)常通过生成中间推理步骤来解决推理问题,这些步骤可视为线性化的搜索树:模型扩展部分解,失败后放弃并回溯尝试其他路径。与传统启发式搜索相比,该策略的优势在于能利用完整的搜索历史而非仅当前局部状态。我们在积木世界、网格导航和推箱子三个控制环境中测试发现,仅提供搜索历史并不能稳定超越启发式搜索。进一步分析表明,问题在于搜索树结构在推理痕迹中是隐式的:当模型回溯或切换分支时,轨迹未明确标识所重访的先前状态。我们提出在线性搜索树(LinTree)中加入简单父指针以显式表达树结构,结果表明,该方法在任务表现和搜索效率上均优于隐式推理模型及基于LLM启发式的搜索。这说明,只有当搜索历史的树形结构被显式建模时,其价值才能充分释放,从而推动更结构感知的大模型推理设计。

原文摘要 · Abstract (English)

Large language models (LLMs) often solve reasoning problems by generating intermediate traces that explore and revise partial solutions. From a search perspective, these traces can be viewed as linearized search trees, where the model extends a partial solution, abandons it when it fails, and backtracks to try alternatives. Compared with traditional heuristic-guided search, such a policy has a potential advantage: it conditions on the whole search trace rather than only on the current local state. We first test whether LLMs utilize this advantage by comparing trace-conditioned reasoning policies against best-first search equipped with an LLM heuristic that only observes the current local state. Across three controlled reasoning environments, Blocks World, grid Navigation, and Sokoban, we find that raw access to search history alone is not enough to reliably outperform heuristic search. We then study one possible reason: in LLM reasoning traces, the underlying search tree is only implicitly represented, and when the model backtracks or switches branches, the trace does not explicitly identify which earlier search state is being revisited. We show that adding simple parent pointers to explicitly represent the linearized tree (LinTree) structure improves both task performance and search efficiency relative to implicit reasoning models and LLM-heuristic-guided search. These results suggest that search history becomes most useful when its tree structure is made explicit, motivating more structure-aware representations for LLM reasoning.

大模型推理搜索树结构化记忆

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。