利用引用关系挖掘生物医学领域难负样本,提升检索模型性能
BiCA: Effective Biomedical Dense Retrieval with Citation-Aware Hard Negatives
- 基于文献引用关系构造难负样本,避免重复但保持语义相关性
- 在BEIR和LoTTE数据集上,nDCG@10与Success@5指标均优于基线
- 仅需少量微调即可实现领域自适应,适合资源受限的科研场景
难负样本对训练高效检索模型至关重要。传统方法依赖交叉编码器或静态嵌入模型,通过余弦距离等相似度度量挖掘难负样本,但在生物医学领域因难以区分源文档与难负样本而面临挑战。本文提出BiCA:基于引用感知的生物医学密集检索方法,利用20,000篇PubMed文章中的引用链接构建难负样本,以提升特定领域小型密集检索器性能。我们使用这些引用引导的负样本微调GTE_small和GTE_Base模型,在BEIR数据集上的in-domain与out-of-domain任务中,零样本检索的nDCG@10均有持续提升;在LoTTE长尾主题任务中,Success@5也超越基线。结果表明,利用文档链接结构生成高信息量负样本,可在极小微调下实现顶尖性能,为数据高效的领域适配提供新路径。
原文摘要 · Abstract (English)
Hard negatives are essential for training effective retrieval models. Hard-negative mining typically relies on ranking documents using cross-encoders or static embedding models based on similarity metrics such as cosine distance. Hard negative mining becomes challenging for biomedical and scientific domains due to the difficulty in distinguishing between source and hard negative documents. However, referenced documents naturally share contextual relevance with the source document but are not duplicates, making them well-suited as hard negatives. In this work, we propose BiCA: Biomedical Dense Retrieval with Citation-Aware Hard Negatives, an approach for hard-negative mining by utilizing citation links in 20,000 PubMed articles for improving a domain-specific small dense retriever. We fine-tune the GTE_small and GTE_Base models using these citation-informed negatives and observe consistent improvements in zero-shot dense retrieval using nDCG@10 for both in-domain and out-of-domain tasks on BEIR and outperform baselines on long-tailed topics in LoTTE using Success@5. Our findings highlight the potential of leveraging document link structure to generate highly informative negatives, enabling state-of-the-art performance with minimal fine-tuning and demonstrating a path towards highly data-efficient domain adaptation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。