arXiv:2501.00332cs.CLcs.IR2025-01ACL被引 86

用多个AI代理协作过滤检索文档,提升问答准确率并减少无效信息。

MAIN-RAG: Multi-Agent Filtering Retrieval-Augmented Generation

  • 多智能体协同评分与动态阈值过滤,无需训练即可提升检索质量。
  • 在4个问答数据集上准确率提升2%-11%,无关文档数量显著减少。
  • 适合追求高可靠、低延迟的实时问答系统开发者使用。

大型语言模型在自然语言处理任务中日益重要,但常生成过时或错误信息。检索增强生成(RAG)通过引入外部实时信息检索来约束模型输出,缓解此问题。然而现有RAG系统常因检索文档质量差而表现不佳:无关或噪声文档会降低性能、增加计算开销并削弱回答可靠性。为此,我们提出无需训练的多智能体过滤检索增强生成框架(MAIN-RAG),利用多个大模型代理协作对检索文档进行过滤与评分。该方法引入自适应过滤机制,根据得分分布动态调整相关性阈值,在保持高召回率的同时有效降低噪声。通过智能体间共识实现鲁棒文档选择,无需额外训练数据或微调。在四个问答基准上的实验表明,MAIN-RAG持续优于传统RAG方法,答案准确率提升2%-11%,同时减少无关检索文档数量。定量分析显示,该方法在响应一致性和答案准确性上均优于基线,为无需训练的实用替代方案。

原文摘要 · Abstract (English)

Large Language Models (LLMs) are becoming essential tools for various natural language processing tasks but often suffer from generating outdated or incorrect information. Retrieval-Augmented Generation (RAG) addresses this issue by incorporating external, real-time information retrieval to ground LLM responses. However, the existing RAG systems frequently struggle with the quality of retrieval documents, as irrelevant or noisy documents degrade performance, increase computational overhead, and undermine response reliability. To tackle this problem, we propose Multi-Agent Filtering Retrieval-Augmented Generation (MAIN-RAG), a training-free RAG framework that leverages multiple LLM agents to collaboratively filter and score retrieved documents. Specifically, MAIN-RAG introduces an adaptive filtering mechanism that dynamically adjusts the relevance filtering threshold based on score distributions, effectively minimizing noise while maintaining high recall of relevant documents. The proposed approach leverages inter-agent consensus to ensure robust document selection without requiring additional training data or fine-tuning. Experimental results across four QA benchmarks demonstrate that MAIN-RAG consistently outperforms traditional RAG approaches, achieving a 2-11% improvement in answer accuracy while reducing the number of irrelevant retrieved documents. Quantitative analysis further reveals that our approach achieves superior response consistency and answer accuracy over baseline methods, offering a competitive and practical alternative to training-based solutions.

检索增强多智能体LLM优化问答系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。