对比五种检索策略，找到生物医学问答中最佳检索方法。
Benchmarking Retrieval Strategies for Biomedical Retrieval-Augmented Generation: A Controlled Empirical Study
- 固定生成模型和向量库，仅比较不同检索策略效果。
- 交叉编码重排序得分最高，上下文精确率达0.852。
- 多查询扩展反而降低精确率，适合需要召回的场景。
检索增强生成（RAG）为大语言模型输出提供外部知识支撑，但在高风险领域如生物医学中，哪种检索策略最优尚未得到充分的控制性实证研究。本文系统比较了五种检索策略——密集向量搜索、混合BM25+密集检索、交叉编码重排序、多查询扩展和最大边际相关性（MMR）——在生物医学问答RAG管道中的表现。所有策略使用相同的生成模型（GPT-4o-mini）、向量存储（ChromaDB）和嵌入模型（text-embedding-3-small），确保差异仅来自检索环节。评估基于预处理后的BioASQ基准子集（rag-mini-bioasq）中的250组问答对，采用四个DeepEval指标：上下文精确率、上下文召回率、忠实度与答案相关性，均报告95%置信区间。包含无上下文对照组作为下界。交叉编码重排序获得最高综合得分（0.827）和最高上下文精确率（0.852），证明查询-文档交互带来显著提升。多查询扩展虽以召回为导向，但上下文精确率最低（0.671），表明盲目扩展查询会引入噪声。MMR以牺牲相关性换取多样性，而密集基线（综合得分0.822）仅比最优策略低0.005。所有RAG方案在答案相关性上（0.658–0.701）远超无上下文对照组（0.287），验证了检索的实际价值。完整流水线、超参数及评估代码已公开。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) offers a well-established path to grounding large language model (LLM) outputs in external knowledge, yet the question of which retrieval strategy works best in a high-stakes domain such as biomedicine has not received the controlled, multi-metric treatment it deserves. This paper presents a systematic empirical comparison of five retrieval strategies -- Dense Vector Search, Hybrid BM25 + Dense retrieval, Cross-Encoder Reranking, Multi-Query Expansion, and Maximal Marginal Relevance (MMR) -- within a biomedical question-answering RAG pipeline. All strategies share a fixed generation model (GPT-4o-mini), a common vector store (ChromaDB), and OpenAI's text-embedding-3-small embeddings, ensuring that observed differences are attributable to retrieval alone. Evaluation is conducted on 250 question-answer pairs drawn from a preprocessed subset of the BioASQ benchmark (rag-mini-bioasq) using four DeepEval metrics: contextual precision, contextual recall, faithfulness, and answer relevancy, each reported with 95% confidence intervals. A no-context ablation is included as a lower bound. Cross-Encoder Reranking achieves the best composite score (0.827) and highest contextual precision (0.852), confirming that query-document interaction yields measurable retrieval gains. Multi-Query Expansion, despite its recall-oriented design, produces the weakest contextual precision (0.671), suggesting naive query diversification introduces retrieval noise. MMR sacrifices answer relevancy for diversity, while the Dense baseline (composite 0.822) falls within 0.005 points of the top strategy. All RAG conditions dramatically outperform the no-context ablation on answer relevancy (0.658-0.701 vs. 0.287), confirming the practical value of retrieval. The full pipeline, hyperparameters, and evaluation code are publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。