arXiv:2605.08887cs.AIcs.CL2026-05被引 1

通过智能采样与知识分组,让多模态智能体高效自进化。

Ace-Skill: Bootstrapping Multimodal Agents with Prioritized and Clustered Evolution

论文配图:Ace-Skill: Bootstrapping Multimodal Agents with Prioritized and Clustered Evolution
图 1 · 摘自论文原文
  • 用优先级采样和动态追踪聚焦低掌握度、高价值任务。
  • 在4个基准上提升35.46%准确率,小模型也能零样本继承能力。
  • 适合想低成本提升智能体持续学习能力的研究者与开发者。

自进化智能体通过将任务交互提炼为可复用的知识资产,有望实现持续适应。但实际中受限于两大耦合瓶颈:数据效率低下,大量计算资源被浪费在低价值样本上;知识干扰,共享库中异构知识导致检索噪声和任务错配。二者形成自我强化的失败循环:低效回放产生噪声知识,进而恶化后续回放。本文提出Ace-Skill,一种协同优化回放分配与知识组织的共进化框架。具体包括:基于优先级的采样器与懒惰衰减熟练度追踪,聚焦于信息量高且未充分掌握的样本;以及语义聚类的知识组织器,实现更清晰的检索与可靠适应。联合优化采样与组织,使自进化转变为良性循环——更优回放生成更高质量知识,反哺更强回放。在四个多模态工具使用基准上,Ace-Skill实现显著提升(如Avg@4准确率相对提高35.46%),使开源35B MoE模型达到或超越专有模型性能。所获知识还能零样本迁移至9B和4B小模型,使资源受限智能体无需额外训练即可获得高级能力。代码已公开于https://github.com/AMAP-ML/Ace-Skill。

原文摘要 · Abstract (English)

Self-evolving agents present a promising path toward continual adaptation by distilling task interactions into reusable knowledge artifacts. In practice, this paradigm remains hindered by two coupled bottlenecks: data inefficiency, where costly rollout effort is disproportionately spent on low-value samples rather than informative ones, and knowledge interference, where heterogeneous knowledge stored in shared repositories leads to noisy retrieval and task-misaligned guidance. Together, these issues form a self-reinforcing failure loop in which uninformative rollouts yield noisy knowledge, which in turn degrades subsequent rollouts. In this work, we introduce Ace-Skill, a co-evolutionary framework that jointly optimizes rollout allocation and knowledge organization for self-evolving multimodal agents. Specifically, Ace-Skill combines aprioritized sampler with lazy-decay proficiency tracking to focus rollouts on informative and insufficiently mastered samples, and a clustered organizer that semantically clusters knowledge for cleaner retrieval and more reliable adaptation. By improving sampling and organization together, Ace-Skill turns self-evolution into a virtuous cycle in which more informative rollouts produce higher-quality knowledge that supports stronger subsequent rollouts. Across four multimodal tool-use benchmarks, Ace-Skill delivers strong gains (e.g., +35.46% relative improvement in Avg@4 accuracy), enabling an opensource 35B MoE model to match or surpass proprietary models. The acquired knowledge also transfers effectively in a zero-shot manner to smaller 9B and 4B models, allowing resource-constrained agents to inherit advanced capabilities without additional training. The code has been publicly available at https://github.com/AMAP-ML/Ace-Skill.

自进化多模态知识管理高效训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。