让多个客户端协作增强问答模型，共享知识库提升低资源表现
CoRAG: Collaborative Retrieval-Augmented Generation
- 多客户端共享检索库，协同训练统一问答模型
- 在少样本场景下超越传统方法，准确率显著提升
- 发现无关文档可能有益，错误样本会降低性能
检索增强生成（RAG）模型在知识密集型任务中表现优异，尤其在少样本学习条件下。我们提出CoRAG框架，将RAG扩展至协作场景，客户端通过共享的段落存储联合训练一个统一模型。为评估CoRAG，我们构建了CRAB基准，用于同质开放域问答的协作评测。实验表明，在低资源场景下，CoRAG持续优于参数化协作学习方法和本地训练的RAG模型。进一步分析揭示：共享存储中相关段落至关重要，出人意料的是，引入无关段落反而有益，而硬负例可能对性能产生负面影响。这为协作RAG带来新思考：如何权衡集体丰富知识库带来的增益与来自其他客户端的有害段落风险。研究结果证实了CoRAG的可行性，同时指出了关键设计挑战与未来研究方向。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) models excel in knowledge-intensive tasks, especially under few-shot learning constraints. We introduce CoRAG, a framework extending RAG to collaborative settings, where clients jointly train a shared model using a collaborative passage store. To evaluate CoRAG, we introduce CRAB, a benchmark for collaborative homogeneous open-domain question answering. Our experiments demonstrate that CoRAG consistently outperforms both parametric collaborative learning methods and locally trained RAG models in low-resource scenarios. Further analysis reveals the critical importance of relevant passages within the shared store, the surprising benefits of incorporating irrelevant passages, and the potential for hard negatives to negatively impact performance. This introduces a novel consideration in collaborative RAG: the trade-off between leveraging a collectively enriched knowledge base and the potential risk of incorporating detrimental passages from other clients. Our findings underscore the viability of CoRAG, while also highlighting key design challenges and promising avenues for future research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。