将推理任务拆给两个协作智能体,提升问答模型的准确性和训练效率。
Divide and Cooperate: Role-Decomposed Multi-Agent LLM Training with Cross-Agent Learning Signals

- 把搜索和生成拆成两个专用智能体,各自用针对性信号训练。
- 实验显示在多跳问答上超越全参数微调的单模型基线。
- 适合需要高可靠性推理的复杂问答场景,尤其关注可解释性。
当前执行多步推理的语言智能体在知识密集型问答中表现优异,但现有方法通常将证据获取与答案生成耦合于单一策略中。这迫使单个模型承担多个可能冲突的角色,导致策略空间呈组合爆炸式增长,阻碍高效探索。同时训练中存在信用分配难题:检索到足够证据的动作,若生成失败仍会被惩罚;反之亦然。本文提出 DAC(Divide and Cooperate)框架,将智能体搜索分解为两个协作子任务,分别由专用智能体处理,并使用角色特异性学习信号进行训练。生成器兼具答案生成与证据充分性验证功能,在证据不足时选择不回答。该不回答信号被纳入搜索智能体的奖励函数,提供结构化的跨智能体学习信号,改善信用分配。同时,搜索者通过硬正例增强,向生成器暴露多样且挑战性的证据环境,提升其鲁棒性。在通用及多跳问答基准上的实验表明,基于共享主干网络、采用参数高效 LoRA 模块实现的 DAC,在性能上优于依赖全参数微调的单体模型基线。
原文摘要 · Abstract (English)
Modern language agents which perform multi-step reasoning have shown strong performance in knowledge-intensive question answering. However, existing approaches typically couple evidence acquisition and answer generation within a single policy. This forces a single model to play multiple potentially conflicting roles, inducing a combinatorial explosion in the policy space and hindering efficient exploration. It also introduces a credit assignment problem during training: a search action that retrieves sufficient evidence may still be penalized when generation fails, and vice versa. We propose DAC (Divide and Cooperate), a role-decomposed multi-agent training framework that divides agentic search into two cooperative subtasks, each handled by a dedicated agent trained with role-specific learning signals. The generator serves a dual role as both an answer producer and an evidence sufficiency verifier, abstaining when retrieved evidence is insufficient. This abstention signal is incorporated into the search agent's reward, providing structured cross-agent learning signals that improve credit assignment. Conversely, the searcher exposes the generator to diverse and challenging evidence environments by hard-positive evidence augmentation, improving its robustness. Experiments on general and multi-hop QA benchmarks show that DAC, implemented via parameter-efficient LoRA modules over a shared backbone, achieves strong performance against prior baselines that rely on full fine-tuning of monolithic models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。