提出语义召回率，更真实评估向量搜索算法性能。
Semantic Recall for Vector Search
- 仅考虑语义相关且理论上可检索的样本，避免误罚无关近邻。
- 在嵌入数据集中，多数查询的最近邻中真正相关样本很少。
- 适用于评估低相关性场景，适合优化检索算法性价比。
我们提出语义召回率（Semantic Recall），一种新指标，用于评估近似最近邻搜索算法的质量，仅关注理论上可通过精确最近邻搜索检索到的语义相关对象。与传统召回率不同，语义召回率不惩罚算法未能检索到与查询语义无关的邻居，即使这些邻居在距离上很近。我们发现，在嵌入数据集中，许多查询的最近邻中真正相关的样本极少，这一情况普遍存在。此外，我们引入容差召回率（Tolerant Recall）作为语义召回率的代理指标，当无法识别语义相关对象时使用。实验证明，该指标更能有效反映检索质量，优化算法以提升该指标可实现更好的成本-质量平衡。
原文摘要 · Abstract (English)
We introduce Semantic Recall, a novel metric to assess the quality of approximate nearest neighbor search algorithms by considering only semantically relevant objects that are theoretically retrievable via exact nearest neighbor search. Unlike traditional recall, semantic recall does not penalize algorithms for failing to retrieve objects that are semantically irrelevant to the query, even if those objects are among their nearest neighbors. We demonstrate that semantic recall is particularly useful for assessing retrieval quality on queries that have few relevant results among their nearest neighbors-a scenario we uncover to be common within embedding datasets. Additionally, we introduce Tolerant Recall, a proxy metric that approximates semantic recall when semantically relevant objects cannot be identified. We empirically show that our metrics are more effective indicators of retrieval quality, and that optimizing search algorithms for these metrics can lead to improved cost-quality tradeoffs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。