提出新型距离度量，让检索更懂语义关系
MA-DPR: Manifold-aware Distance Metrics for Dense Passage Retrieval
- 用近邻图建模文本嵌入的非线性结构
- 在分布外数据上性能提升最高达26%
- 适合需要强泛化能力的检索场景
密集段落检索（DPR）通常依赖欧氏或余弦距离衡量嵌入空间中的查询-段落相关性，这在嵌入位于线性流形时有效。然而，我们在多个DPR基准上的实验表明，嵌入往往位于低维非线性流形上，尤其是在分布外（OOD）设置下，余弦和欧氏距离无法捕捉语义相似性。为此，我们提出了面向流形的距离度量（MA-DPR），通过近邻图建模段落的内在流形结构，并基于图中最短路径计算查询-段落距离。实验显示，MA-DPR在分布外段落检索中相比欧氏与余弦距离性能提升最高达26%，且在多种嵌入模型上保持相近的分布内表现，仅带来极小的查询推理时间增加。实证结果表明，流形感知距离使DPR能利用相关邻近段落的上下文信息，即使缺乏直接语义重叠也有效。MA-DPR可应用于广泛的稠密嵌入与检索任务，在多个领域具有潜在优势。
原文摘要 · Abstract (English)
Dense Passage Retrieval (DPR) typically relies on Euclidean or cosine distance to measure query-passage relevance in embedding space, which is effective when embeddings lie on a linear manifold. However, our experiments across DPR benchmarks suggest that embeddings often lie on lower-dimensional, non-linear manifolds, especially in out-of-distribution (OOD) settings, where cosine and Euclidean distance fail to capture semantic similarity. To address this limitation, we propose a manifold-aware distance metric for DPR (MA-DPR) that models the intrinsic manifold structure of passages using a nearest neighbor graph and measures query-passage distance based on their shortest path in this graph. We show that MA-DPR outperforms Euclidean and cosine distances by up to 26% on OOD passage retrieval with comparable in-distribution performance across various embedding models while incurring a minimal increase in query inference time. Empirical evidence suggests that manifold-aware distance allows DPR to leverage context from related neighboring passages, making it effective even in the absence of direct semantic overlap. MADPR can be applied to a wide range of dense embedding and retrieval tasks, offering potential benefits across a wide spectrum of domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。