让语言模型自我进化,用知识图谱存经验并持续优化推理能力。
MAGE: Multi-Agent Self-Evolution with Co-Evolutionary Knowledge Graphs

- 构建四子图共进化知识图谱,外化存储学习经验与纠错信息。
- 在9个任务上超越提示冻结基线,数学与多跳问答表现最优。
- 适合需要长期推理积累的智能体系统,如复杂决策或开放世界任务。
自进化语言模型需决定学什么以及如何在迭代中保留已学内容。现有系统通常以自然语言反馈、扁平记忆或隐式强化信号携带跨轮次知识,均难以支持推理模型在推理时保持冻结。本文提出MAGE(多智能体图引导进化)框架,将自知识外化为四子图共进化知识图谱。其经验子图同时存储教师编写的失败修正和学习者自身的历史正确推理轨迹,作为任务条件引导,供冻结执行模型使用。演化过程中,图结构、任务级搜索赌徒与技能级路由赌徒从同一奖励流更新,而学习者主干保持不变。结构分析表明,仅追加的记忆增长、有限的课程覆盖范围及任务过滤检索共同支撑了冻结学习者演化的稳定提升。在涵盖数学推理、多跳与开放域问答、时空分析、金融数值推理、医学选择题、开放世界生存游戏和网页导航等九项基准测试中,MAGE性能显著优于基于提示的冻结主干基线。消融实验显示,自获取的成功轨迹与教师编写修正互补:成功记忆在模板依赖型任务中贡献更大,修正记忆则更利于复杂组合与交互场景。
原文摘要 · Abstract (English)
Self-evolving language-model agents must decide what to learn next and how to preserve what they have learned across iterations. Existing systems typically carry this cross-iteration knowledge as natural-language feedback, flat episodic memory, or implicit reinforcement signals, none of which cleanly supports a frozen weak backbone at inference time. This paper introduces MAGE (Multi-Agent Graph-guided Evolution), a framework that externalizes self-knowledge into a four-subgraph co-evolutionary knowledge graph. Its experience subgraph stores both teacher-written failure corrections and the learner's own past correct reasoning traces, which are retrieved as task-conditioned guidance for a frozen execution model. During evolution, the graph, a task-level search bandit, and a skill-level routing bandit are updated from the same reward stream, while the learner's backbone remains unchanged. We further provide structural analysis showing how append-only memory growth, bounded curriculum coverage, and task-filtered retrieval together support stable improvement of the retrieval substrate for frozen-learner evolution. Across nine benchmarks spanning mathematical reasoning, multi-hop and open-domain question answering, spatio-temporal analysis, financial numerical reasoning, medical multiple-choice, an open-world survival game, and web navigation, MAGE achieves strong performance against prompt-based frozen-backbone baselines. Ablations show that self-harvested success traces and teacher-written corrections are complementary, with success memories contributing most on reasoning-template-heavy tasks and corrective memories supporting harder composition and interaction settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。