让大模型的规划与执行更协调，提升工具调用能力
Capability-Aligned Hierarchical Learning for Tool-Augmented LLMs
- 用强化学习联合优化高层规划与底层执行策略
- 在三个测试集上显著提升工具使用准确率
- 适合需要精准任务分解的智能助手场景
工具学习使大语言模型能够调用外部工具完成任务。以往研究采用分层结构：高层策略负责全局规划并分解任务，底层策略专注调用工具解决子任务。然而，这些方法通常分别优化上下层策略，导致规划与执行不匹配，限制了模型在工具使用任务上的表现。本文提出能力对齐的分层学习（CAHL），利用强化学习与价值回归（RLVR）联合优化两层策略，增强高层规划者与底层执行者之间的对齐。在受限工具使用基准（API-Bank 和 BFCL）及开放环境（Bamboogle）上的实验表明，CAHL 方法有效提升了模型性能。
原文摘要 · Abstract (English)
Tool learning enables LLMs to invoke external tools to accomplish tasks. Prior studies have demonstrated the effectiveness of a hierarchical structure: a high-level policy handles global planning and decomposes tasks into manageable sub-tasks, and a low-level policy focuses on invoking tools to solve these sub-tasks. However, these works typically optimize the high-level and low-level policies separately, leading to planner-executor misalignment and limiting LLM performance on tool-use tasks. In this paper, we propose a method called Capability-Aligned Hierarchical Learning (CAHL), which leverages RLVR to jointly optimize both policies, enabling better alignment between the high-level planner and the low-level executor. Experiments on constrained tool-use benchmarks (API-Bank and BFCL) and an open-ended environment (Bamboogle) demonstrate the effectiveness of CAHL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。