用轻量预测器让大模型推理更省力又准确。
Domain-Specialized Tree of Thought through Plug-and-Play Predictors
- 引入可插拔预测器,动态指导搜索路径选择。
- 在多个任务上准确率超基线,计算量降低26%~75%。
- 适合需要高效复杂推理的实用场景,如数学与逻辑题。
尽管大语言模型在复杂推理方面取得进展,但主流方法如思维树(ToT)框架面临探索深度与计算效率之间的权衡。现有实现常依赖重型基于LLM的自评估或僵化启发式规则进行分支剪枝,导致成本过高且缺乏灵活性。为此,我们提出DST,一种可适应、即插即用的预测器,作为轻量级监督启发式方法,引导ToT搜索过程。该预测器实现动态、上下文感知的剪枝,使搜索在简单推理步骤中近乎贪婪高效,仅在遇到不确定或任务复杂时自适应扩展搜索宽度。我们在涵盖数学推理、通用推理和复杂逻辑推理的多样化基准上评估该方法。实验结果表明,本方法在准确率上达到或优于强基线(包括标准ToT),同时计算开销减少26%-75%。本工作有效解决树状推理中的准确率-效率权衡问题,将ToT从资源密集型技术转变为适用于大模型复杂问题求解的可扩展、实用范式。
原文摘要 · Abstract (English)
While Large Language Models (LLMs) have advanced complex reasoning, prominent methods like the Tree of Thoughts (ToT) framework face a critical trade-off between exploration depth and computational efficiency. Existing ToT implementations often rely on heavyweight LLM-based self-evaluation or rigid heuristics for branch pruning, making them prohibitively expensive and inflexible for broad application. To address this, we introduce DST, an adaptable, plug-and-play predictor that serves as a lightweight, supervised heuristic to guide the ToT search process. Our predictor enables dynamic, context-aware pruning, allowing the search to proceed with near-greedy efficiency on simpler reasoning steps while adaptively expanding the search beam only when encountering uncertainty or task complexity. We evaluate our approach on a diverse suite of benchmarks spanning mathematical reasoning, general reasoning, and complex logical reasoning. Experimental results demonstrate that our method achieves accuracy competitive with or superior to strong baselines, including standard ToT, while reducing computational overhead by 26-75%. Our work effectively resolves the accuracy-efficiency trade-off in tree-based reasoning, transforming ToT from a resource-intensive technique into a scalable and practical paradigm for complex problem-solving in LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。