对比8个领域对话问答数据集，发现简单检索方法更有效
Comprehensive Comparison of RAG Methods Across Multi-Domain Conversational QA
- 统一实验设置下比较8种RAG方法在多轮对话中的表现
- 重排、混合BM25和HyDE等简单方法优于复杂模型，部分先进方法反而更差
- 检索效果受数据集特性和对话长度影响大，需匹配策略与数据结构
对话问答日益依赖检索增强生成（RAG）将大语言模型（LLMs）与外部知识结合。然而，现有研究多孤立评估RAG方法，且集中于单轮场景。本文填补了多轮对话问答中RAG方法系统性比较的空白，其中对话历史、指代消解和用户意图变化显著增加检索难度。我们在八个跨领域的多样化对话问答数据集上，对基础与先进RAG方法进行了全面实证研究。采用统一实验设置，评估检索质量与答案生成性能，分析各对话轮次的表现演变。结果表明，稳健而简单的策略如重排、混合BM25和HyDE，始终优于原始RAG；而若干先进方法不仅未提升性能，甚至低于无RAG基线。我们进一步证明，数据集特性与对话长度强烈影响检索有效性，解释了为何单一RAG策略无法在所有场景占优。总体而言，有效对话RAG的关键不在于方法复杂度，而在于检索策略与数据结构的匹配。代码已公开。
原文摘要 · Abstract (English)
Conversational question answering increasingly relies on retrieval-augmented generation (RAG) to ground large language models (LLMs) in external knowledge. Yet, most existing studies evaluate RAG methods in isolation and primarily focus on single-turn settings. This paper addresses the lack of a systematic comparison of RAG methods for multi-turn conversational QA, where dialogue history, coreference, and shifting user intent substantially complicate retrieval. We present a comprehensive empirical study of vanilla and advanced RAG methods across eight diverse conversational QA datasets spanning multiple domains. Using a unified experimental setup, we evaluate retrieval quality and answer generation using generator and retrieval metrics, and analyze how performance evolves across conversation turns. Our results show that robust yet straightforward methods, such as reranking, hybrid BM25, and HyDE, consistently outperform vanilla RAG. In contrast, several advanced techniques fail to yield gains and can even degrade performance below the No-RAG baseline. We further demonstrate that dataset characteristics and dialogue length strongly influence retrieval effectiveness, explaining why no single RAG strategy dominates across settings. Overall, our findings indicate that effective conversational RAG depends less on method complexity than on alignment between the retrieval strategy and the dataset structure. We publish the code used.\footnote{\href{https://github.com/Klejda-A/exp-rag.git}{GitHub Repository}}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。