用密集检索提升对话搜索准确率，无需大量调优。
Dense Passage Retrieval in Conversational Search
- 采用双编码器生成上下文嵌入，实现高效索引与聚类。
- 在CAsT数据集上，密集检索超越BM25，无需精细微调。
- 适合研究对话系统检索优化的开发者与研究人员。
信息检索系统传统上依赖于精确词项匹配方法（如BM25）进行第一阶段检索。然而，近年来基于神经网络的技术引入了一种新方法——密集检索。该方法使用双编码器生成上下文嵌入，可在运行时高效索引和聚类，显著提升开放域问答系统的检索性能。本文将密集检索技术应用于对话搜索，在CAsT基准数据集上开展实验，并提出一个端到端的对话搜索系统GPT2QR+DPR，结合多种查询重写策略以提高检索准确率。结果表明，即使不进行大量微调，密集检索仍优于BM25。本工作推动了基于神经网络的检索方法在对话搜索中的研究进展，凸显了密集检索在提升对话搜索系统检索精度方面的潜力。
原文摘要 · Abstract (English)
Information retrieval systems have traditionally relied on exact term match methods such as BM25 for first-stage retrieval. However, recent advancements in neural network-based techniques have introduced a new method called dense retrieval. This approach uses a dual-encoder to create contextual embeddings that can be indexed and clustered efficiently at run-time, resulting in improved retrieval performance in Open-domain Question Answering systems. In this paper, we apply the dense retrieval technique to conversational search by conducting experiments on the CAsT benchmark dataset. We also propose an end-to-end conversational search system called GPT2QR+DPR, which incorporates various query reformulation strategies to improve retrieval accuracy. Our findings indicate that dense retrieval outperforms BM25 even without extensive fine-tuning. Our work contributes to the growing body of research on neural-based retrieval methods in conversational search, and highlights the potential of dense retrieval in improving retrieval accuracy in conversational search systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。