按任务难易动态分配模型,让推理更省token还更准
RouteGoT: Node-Adaptive Routing for Cost-Efficient Graph of Thoughts Reasoning
- 根据节点难度自动选强/弱模型,重点任务用强模型,简单任务用轻量方案
- 相比AGoT提升8.1%准确率,输出token减少79.1%
- 支持用户设定预算,适合追求性价比的复杂推理场景
大型语言模型在多步推理中表现优异,但增加推理结构复杂度并不总能带来系统级收益。树状思维(ToT)、图状思维(GoT)及自适应图状思维(AGoT)虽在部分基准上提升准确率,却常导致显著的令牌消耗和延迟,且性能在不同任务分布间不稳定,有时甚至不如链式思维(CoT)或直接输入输出提示(IO)。我们归因于GoT类推理流程中阶段与节点层面的异质性:高质量规划与最终合成全局耦合,需强模型支持;而多数中间子任务局部性强,可用轻量模型以极少令牌完成。受此启发,我们提出RouteGoT——一种可控制预算、节点自适应路由的图结构推理框架。它通过优先将强模型用于规划与合成,动态为叶节点子任务分配轻量模型与低成本策略,并将显式预算约束集成至全局调度器中,实现用户指定令牌预算下的图结构扩展控制,从而获得可预测的性能-成本权衡。在推理、检索及多跳问答等基准上的实验表明,RouteGoT在匹配或超越现有方法准确率的同时,大幅降低令牌使用量;相较AGoT,平均提升8.1个百分点准确率,输出令牌减少79.1%。此外,其在不同预算目标与任务下均保持更优的成本-准确率平衡,展现更强鲁棒性。
原文摘要 · Abstract (English)
Large Language Models (LLMs) excel at multi-step reasoning, yet increasing the structural complexity of inference does not consistently improve system-level returns. Methods such as Tree of Thoughts (ToT), Graph of Thoughts (GoT), and Adaptive Graph of Thoughts (AGoT) can boost accuracy on some benchmarks, but often introduce substantial overhead in token consumption and latency, and their gains can be unstable across task distributions-sometimes underperforming simpler Chain-of-Thought (CoT) or direct input-output prompting (IO). We attribute this inefficiency to stage-wise and node-wise heterogeneity inside GoT-style reasoning pipelines: high-quality planning and final synthesis are globally coupled and typically benefit from strong models, whereas many intermediate subtasks are localized and can be solved accurately by lighter models with far fewer tokens. Motivated by these observations, we propose RouteGoT, a budget-controllable, node-adaptive routing framework for graph-structured reasoning. RouteGoT performs in-graph routing by prioritizing strong models for planning and synthesis, while dynamically allocating lightweight models and cost-effective strategies to leaf subtasks based on predicted difficulty. It further integrates explicit budget constraints into a global inference scheduler to control graph expansion under a user-specified token budget, enabling predictable performance-cost trade-offs. Experiments across reasoning, retrieval, and multi-hop QA benchmarks show that RouteGoT matching or improving accuracy while substantially reducing token usage; specifically, it achieves an average 8.1 percentage points accuracy improvement and 79.1\% output token reduction compared to AGoT. Furthermore, RouteGoT outperforms existing routing baselines by maintaining a superior cost-accuracy trade-off, demonstrating improved robustness under varying budget targets and tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。