用相似查询样例增强检索，让编码器模型表现更好。
RARe: Retrieval Augmented Retrieval with In-Context Examples
- 用语义相似的查询-文档对作为上下文示例微调模型。
- 在BeIR和RAR-b数据集上最高提升2.72% nDCG。
- 适合关注检索模型泛化能力的研究者。
尽管上下文学习在仅解码器语言模型中研究充分，但在仅编码器模型中的应用仍不充分。本文研究了仅编码器模型在文本检索任务中的上下文学习。将与目标查询语义相似的查询-文档对作为上下文示例加入目标查询，能否提升检索器性能？我们提出的RARe方法通过微调预训练模型实现此目标，使用与目标查询语义相近的上下文示例。该方法在开放域检索数据集（BeIR、RAR-b）上相较仅输入目标查询，性能最高提升+2.72% nDCG。尤其发现，相较于未使用上下文示例的模型，RARe展现出更强的跨领域泛化能力，类似大语言模型中的上下文学习现象。我们还对上下文示例增强的设计选择进行了分析，并为未来工作奠定基础。
原文摘要 · Abstract (English)
While in-context learning is well-studied with decoder-only language models (LLMs), its utility for encoder-only models remains underexplored. We study in-context learning for encoder-only models for text retrieval tasks. Can incorporating in-context examples (query-document pairs) to the target query enhance retriever performance? Our approach, RARe, finetunes a pre-trained model with in-context examples whose query is semantically similar to the target query. This approach achieves performance gains of up to +2.72% nDCG across open-domain retrieval datasets (BeIR, RAR-b) compared to using the target query only as an input. In particular, we find RARe exhibits stronger out-of-domain generalization compared to models using queries without in-context examples, similar to what is seen for in-context learning in LLMs. We further provide analysis on the design choices of in-context example augmentation for retrievers and lay the foundation for future work.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。