对比不同检索策略在科学问答中的表现，发现混合检索最稳定。
SciRet: A Compute-Aware Empirical Study of Retrieval and Reranking for Scientific RAG
- 用混合检索融合稀疏与稠密方法，提升科学文献问答的召回率。
- 1K和15K论文规模下，召回率@10达1.000，但跨领域重排序器反而降低精度。
- 结果对研究者有参考价值，尤其关注科学RAG系统构建与计算成本平衡。
我们提出SciRet，一项针对CORD-19数据集的科学问答中检索增强生成的计算感知实证研究。不提出新模型，而是固定一个科学RAG流程，在三个语料规模下评估：1,034个片段（1K篇论文）、5,160个片段（5K篇）和15,480个片段（15K篇）。该流程包括句子窗口分块、BM25、BGE-M3稠密检索、倒数排名融合、可选交叉编码器重排序及基于证据的答案生成。实验表明，混合检索在本设置中比纯稀疏或纯稠密检索更鲁棒，在1K和15K规模下均达到Recall@10为1.000。而使用MS MARCO训练的交叉编码器重排序器反而降低精度，说明领域错配可能抵消强查询-段落交互的优势。在本设置中，随着语料规模增大，生成内容的忠实度（RAGAS）也提升。检索评估使用由混合系统生成的伪相关标签，因此结果视为受控比较证据而非基准声明。代码、索引和评估输出已公开，支持复现与后续研究。
原文摘要 · Abstract (English)
We introduce SciRet, a compute-aware empirical study of retrieval-augmented generation for scientific question answering over CORD-19. Rather than proposing a new model, we evaluate a fixed scientific RAG pipeline across three corpus scales: 1,034 chunks (1K papers), 5,160 chunks (5K papers), and 15,480 chunks (15K papers). The pipeline combines sentence-window chunking, BM25, BGE-M3 dense retrieval, reciprocal rank fusion, optional cross-encoder reranking, and grounded answer generation. Across these settings, hybrid retrieval is more robust than either sparse-only or dense-only retrieval in our setting, reaching Recall@10 of 1.000 at 1K and 15K. In contrast, an MS MARCO-trained cross-encoder reranker reduces precision on the scientific corpus, suggesting that domain mismatch can outweigh the benefits of stronger query-passage interaction. Generation faithfulness measured with RAGAS increases with corpus scale in our setup. Retrieval evaluation uses pseudo-relevance labels derived from the hybrid system, so we treat the results as controlled comparative evidence rather than a benchmark claim. We release code, indexes, and evaluation outputs to support replication and follow-up studies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。