arXiv:2409.13202cs.CL2024-09AAAI被引 7

让大模型用工具更准,还不丢通用能力。

CITI: Enhancing Tool Utilizing Ability in Large Language Models without Sacrificing General Performance

  • 按组件重要性分层训练,关键部分用LoRA微调
  • 在多个工具任务上提升使用能力,通用性能下降少于5%
  • 适合需要强工具调用又怕模型变笨的开发者

工具学习使大语言模型通过调用外部工具与环境交互,提升准确性和能力范围。然而,以往工作主要关注提升工具调用准确率和对新工具的泛化能力,过度强制模型适应特定调用模式,忽视了对模型通用性能的损害。这偏离了引入工具的初衷。为解决此问题,我们通过分析隐藏表示变化和梯度重要性得分,剖析了能力权衡机制。基于分析结果,提出组件重要性驱动的工具使用能力注入方法(CITI)。根据各组件的梯度重要性,对不同组件采用差异化的训练策略:对重要组件使用混合LoRA(MOLoRA),对少数被认为不重要的骨干组件参数进行微调,其余参数保持冻结。实验表明,CITI能有效提升模型工具使用能力,同时将通用性能损失控制在较低水平,在多种评估指标上表现优异。

原文摘要 · Abstract (English)

Tool learning enables the Large Language Models (LLMs) to interact with the external environment by invoking tools, enriching the accuracy and capability scope of LLMs. However, previous works predominantly focus on improving model's tool-utilizing accuracy and the ability to generalize to new, unseen tools, excessively forcing LLMs to adjust specific tool-invoking pattern without considering the harm to model's general performance. This deviates from the actual applications and original intention of integrating tools to enhance model. To tackle this problem, we dissect the capability trade-offs by examining the hidden representation changes and the gradient-based importance score of model's components. Based on the analysis result, we propose a Component Importance-based Tool-utilizing ability Injection method (CITI). According to the gradient-based importance score of different components, it alleviates the capability conflicts caused by fine-tuning process by applying distinct training strategies to different components. CITI applies Mixture-Of-LoRA (MOLoRA) for important components. Meanwhile, it fine-tunes the parameters of few components deemed less important in the backbone of the LLM, while keeping other parameters frozen. CITI can effectively enhance the model's tool-utilizing capability without excessively compromising its general performance. Experimental results demonstrate that our approach achieves outstanding performance across a range of evaluation metrics.

工具调用大模型微调优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。