融合稀疏与稠密检索，提升新冠文献搜索相关性与速度
Hybrid Retrieval for COVID-19 Literature: Comparing Rank Fusion and Projection Fusion with Diversity Reranking
- 结合稀疏、稠密检索与两种融合策略，优化文献召回效果
- 排名融合（RRF）在专家查询上达nDCG@10=0.828，优于纯稠密模型6.1%
- 投影融合（B5）提速33%且多样性更高，适合对速度敏感场景
我们提出一个混合检索系统用于新冠科学文献搜索，在TREC-COVID基准（171,332篇论文，50个专家查询）上评估。系统包含六种配置：稀疏（SPLADE）、稠密（BGE）、排名级融合（RRF）及基于投影的向量融合（B5）。RRF在相关性上表现最佳（nDCG@10 = 0.828），比纯稠密模型高6.1%，比纯稀疏模型高14.9%。投影融合版本在专家查询中实现nDCG@10 = 0.678，速度提升33%（847毫秒 vs. 1271毫秒），且ILD@10高出2.2倍。在400个查询（含专家、机器生成及三种改写形式）上，B5在关键词密集改写中相对提升达+8.8%，但RRF绝对性能仍最优。在专家查询中，MMR重排序使列表内多样性提升23.8%-24.5%，代价为nDCG@10下降20.4%-25.4%。所有查询集下延迟均低于2秒。系统已部署为基于Pinecone无服务器索引的Streamlit应用。
原文摘要 · Abstract (English)
We present a hybrid retrieval system for COVID-19 scientific literature, evaluated on the TREC-COVID benchmark (171,332 papers, 50 expert queries). The system implements six retrieval configurations spanning sparse (SPLADE), dense (BGE), rank-level fusion (RRF), and a projection-based vector fusion (B5) approach. RRF fusion achieves the best relevance (nDCG@10 = 0.828), outperforming dense-only by 6.1% and sparse-only by 14.9%. Our projection fusion variant reaches nDCG@10 = 0.678 on expert queries while being 33% faster (847 ms vs. 1271 ms) and producing 2.2x higher ILD@10 than RRF. Evaluation across 400 queries -- including expert, machine-generated, and three paraphrase styles -- shows that B5 delivers the largest relative gain on keyword-heavy reformulations (+8.8%), although RRF remains best in absolute nDCG@10. On expert queries, MMR reranking increases intra-list diversity by 23.8-24.5% at a 20.4-25.4% nDCG@10 cost. Both fusion pipelines evaluated for latency remain below the sub-2 s target across all query sets. The system is deployed as a Streamlit web application backed by Pinecone serverless indices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。