让多个智能体自动匹配任务并动态调整合作方式，提升协作效率。
SkillGraph: Self-Evolving Multi-Agent Collaboration with Multimodal Graph Topology
- 用多模态图网络动态生成智能体协作关系，根据任务内容实时调整。
- 在四个基准上表现优于传统方法，跨不同系统结构均有效提升。
- 支持智能体自我进化技能库，适合复杂视觉任务的多智能体系统。
将视觉语言模型扩展为视觉多智能体系统（VMAS）面临两大耦合问题：一是通信拓扑在推理前固定，无法感知视觉内容与查询上下文；二是智能体推理能力在部署中保持静态。二者相互加剧：僵化的拓扑难以利用更丰富的智能体专长，而静态智能体也缺乏专业化动力。为此提出SkillGraph，一个联合框架，实现智能体专长与通信拓扑的共同演化。其中，多模态图变换器（MMGT）融合视觉标记、指令语义与活跃技能嵌入，预测依赖查询的协作图，以动态、内容感知的信息流替代人工路由。同时，技能设计师从失败案例中提炼并优化推理启发式，构建可自我演化的多模态技能库。关键在于，更新后的技能嵌入反馈至MMGT，使拓扑随能力增长而自适应调整。实验表明，SkillGraph在四个基准、五种常见多智能体结构和四种基础模型上均实现持续提升。代码已开源。
原文摘要 · Abstract (English)
Scaling vision-language models into Visual Multiagent Systems (VMAS) is hindered by two coupled issues. First, communication topologies are fixed before inference, leaving them blind to visual content and query context; second, agent reasoning abilities remain static during deployment. These issues reinforce each other: a rigid topology fails to leverage richer agent expertise, while static agents lack incentives to specialize for a given query. We address this with SkillGraph, a joint framework that evolves both agent expertise and communication topology. Within this framework, a Multimodal Graph Transformer (MMGT) encodes visual tokens, instruction semantics and active skill embeddings to predict a query-conditioned collaboration graph, replacing hand-crafted routing with dynamic, content-aware information flow. Complementing this, a Skill Designer distills and refines reasoning heuristics from failure cases, constructing a self-evolving multimodal Skill Bank. Crucially, updated skill embeddings are fed back into the MMGT, enabling the topology to adapt alongside capability growth. Experiments show that SkillGraph achieves consistent improvements across four benchmarks, five common MAS structures and four base models. Code is available at https://github.com/niez233/skillgraph.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。