arXiv:2510.00023cs.AI2025-10被引 3

让大模型学会灵活用工具,自动给行为打分,轻松适配具体任务。

ToolBrain: A Flexible Reinforcement Learning Framework for Agentic Tools

  • 用可自定义奖励或AI评判生成奖励,无需人工设计
  • 支持多种训练策略,提升多工具选择与执行效率
  • 适合想快速部署智能代理的研究者和开发者

有效使用工具对智能体至关重要,但现有方法因依赖人工设计奖励、训练数据有限、多工具选择能力差,导致适应慢、资源浪费、性能不佳。我们提出 ToolBrain,一个轻量级、易用的强化学习框架,支持 LLM-based 智能体的工具使用训练。它兼容 GRPO、DPO 等强化学习算法及监督学习,支持直接基于执行轨迹定义奖励,或通过 LLM-as-a-judge 自动生成奖励。框架内置知识蒸馏、从工具描述自动生成任务、无缝工具检索、通过 Unsloth 实现 QLoRA 高效微调,以及 bitsandbytes 的量化推理。在邮件搜索智能体案例中,验证了其在真实工作流下显著提升工具使用能力,代码简洁且可扩展。项目已开源:https://toolbrain.org/。

原文摘要 · Abstract (English)

Effective tool use is essential for agentic AI, yet training agents to utilize tools remains challenging due to manually designed rewards, limited training data, and poor multi-tool selection, resulting in slow adaptation, wasted computational resources, and suboptimal performance. We introduce ToolBrain, a lightweight and user-friendly framework for training tool use in agentic models with flexible reinforcement learning, thereby easing the barriers for researchers and practitioners to adapt LLM-based agents to specific domains. It supports a wide range of training strategies, including reinforcement learning algorithms such as GRPO and DPO, as well as supervised learning. ToolBrain enables custom reward callables directly on an agent's execution traces or simply utilizes an automated LLM-as-a-judge system for reward generation. It is packed with useful capabilities, including knowledge distillation from large to small models, automatic task generation from tool descriptions, seamless tool retrieval, efficient fine-tuning pipelines with QLoRA through Unsloth, and quantized inference via bitsandbytes. We demonstrate ToolBrain through an Email Search Agent case study, showing measurable improvements in tool-use skills under a realistic workflow, while keeping the codebase simple and extensible. Our framework is publicly available at https://toolbrain.org/.

智能体强化学习工具使用LLM应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。