让搜索智能体与知识图谱共同进化,边搜边学边存知识。
CoEvoKG: Co-Evolving Knowledge Graphs with Self-Evolving Search Agents

- 用知识图谱生成多跳问题,同时作为搜索路径的可信证据库。
- 成功搜索后将新知识写回图谱,提升后续任务准确率11.6个百分点。
- 适合想做智能体自进化和知识积累的研究者,尤其关注问答系统。
大语言模型可通过强化学习提升搜索智能体性能,但现有自对弈方法在重复生成任务时会丢弃成功搜索中获得的知识。我们提出CoEvoKG框架,将知识图谱同时作为可验证训练任务的来源和持久化的证据记忆库。该框架联合训练任务生成器与搜索智能体:生成器从知识图谱中采样实体链生成多跳问题,智能体则根据答案正确性及受图谱支持的实体路径获得奖励。当搜索成功时,CoEvoKG验证并去重检索到的证据,将其写回对应图节点与边。后续轮次复用此增强后的图谱进行任务生成与奖励计算,实现模型自我演化与知识累积的闭环。在六个问答基准(NQ、TriviaQA、PopQA、HotpotQA、2WikiMultiHopQA、Bamboogle)上,使用三个主干模型的实验表明,CoEvoKG相较基础模型分别提升宏平均准确率11.2、10.1和11.6个百分点(对应Qwen2.5-3B-Instruct、Qwen2.5-7B-Instruct、Llama-3.1-8B-Instruct)。在相同训练预算下,相较于竞争性自对弈与强化学习基线,进一步提升2.6至3.7个宏平均点。代码已开源。
原文摘要 · Abstract (English)
Large language models can improve with reinforcement learning for search agents, yet existing self play agents repeatedly generate tasks while discarding the knowledge gained during successful searches. We introduce CoEvoKG, a framework that turns a knowledge graph into both a source of verifiable training tasks and a persistent evidence memory for agent evolution. CoEvoKG jointly trains a task generator and a search agent: the generator creates multihop questions from entity chains sampled from the knowledge graph, while the agent learns from rewards for answer correctness and search trajectories whose entity paths are supported by graph evidence. When a search succeeds, CoEvoKG verifies and deduplicates the retrieved evidence, then writes it back to the corresponding graph nodes and edges. Future rounds reuse this enriched graph for task generation and reward computation, closing the loop between model self evolution and knowledge accumulation. Experiments on six QA benchmarks (NQ, TriviaQA, PopQA, HotpotQA, 2WikiMultiHopQA, and Bamboogle) with three backbone models show that CoEvoKG improves macro average accuracy over the corresponding base models by +11.2, +10.1, and +11.6 points on Qwen2.5-3B-Instruct, Qwen2.5-7B-Instruct, and Llama-3.1-8B-Instruct, respectively. Under matched training budgets, CoEvoKG further improves over competitive self play baselines and RL baselines for search agents by +2.6 to +3.7 macro average points across the three backbones. Code is available at https://github.com/lazzy1225/CoEvoKG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。