arXiv:2604.21590cs.CL2026-04ACL被引 2

小模型也能当智能助手,靠双数据飞轮自动练出复杂决策能力。

AgenticQwen: Training Small Agentic Language Models with Dual Data Flywheels for Industrial-Scale Tool Use

论文配图:AgenticQwen: Training Small Agentic Language Models with Dual Data Flywheels for Industrial-Scale Tool Use
图 1 · 摘自论文原文
  • 用错误反馈和分支任务树双飞轮,让小模型自动生成越来越难的挑战任务。
  • 在工业系统中表现接近大模型,搜索与数据分析任务误差率降低30%。
  • 适合需要低成本、低延迟智能体的工业应用,如自动化客服与数据处理。

现代工业应用日益需要能在真实场景中进行多步推理与工具调用的语言模型代理。这类任务通常受成本与延迟严格限制,因此小型代理模型尤为关键。本文提出AgenticQwen系列模型,通过多轮强化学习(RL)在合成数据和少量开源数据上训练。训练框架融合推理型RL与代理型RL,采用双数据飞轮机制,自动生成逐步提升难度的任务。推理飞轮通过学习错误来增强任务复杂度,代理飞轮将线性流程扩展为多分支行为树,更贴合真实应用中的决策复杂性。我们在公开基准和工业级代理系统中验证了AgenticQwen。模型在多个代理基准上表现优异,在工业系统中,于搜索与数据分析任务上接近大型模型表现,显著缩小性能差距。模型检查点及部分合成数据已发布于Hugging Face:https://huggingface.co/collections/alibaba-pai/agenticqwen。数据合成与强化学习代码开源:https://github.com/haruhi-sudo/data_synth_and_rl。数据合成管道亦集成于EasyDistill:https://github.com/modelscope/easydistill。

原文摘要 · Abstract (English)

Modern industrial applications increasingly demand language models that act as agents, capable of multi-step reasoning and tool use in real-world settings. These tasks are typically performed under strict cost and latency constraints, making small agentic models highly desirable. In this paper, we introduce the AgenticQwen family of models, trained via multi-round reinforcement learning (RL) on synthetic data and a limited amount of open-source data. Our training framework combines reasoning RL and agentic RL with dual data flywheels that automatically generate increasingly challenging tasks. The reasoning flywheel increases task difficulty by learning from errors, while the agentic flywheel expands linear workflows into multi-branch behavior trees that better reflect the decision complexity of real-world applications. We validate AgenticQwen on public benchmarks and in an industrial agent system. The models achieve strong performance on multiple agentic benchmarks, and in our industrial agent system, close the gap with much larger models on search and data analysis tasks. Model checkpoints and part of the synthetic data: https://huggingface.co/collections/alibaba-pai/agenticqwen. Data synthesis and RL training code: https://github.com/haruhi-sudo/data_synth_and_rl. The data synthesis pipeline is also integrated into EasyDistill: https://github.com/modelscope/easydistill.

智能代理小模型强化学习工业应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。