通过投影将任意距离转为超度量空间,实现高效近似向量搜索。
Infinity Search: Approximate Vector Search with Projections on q-Metric Spaces
- 用投影将普通距离转为超度量空间,保持最近邻不变。
- 学习投影近似,使查询与数据点间距离计算更快。
- 支持多种参数q的度量空间,可调速与精度平衡。
超度量空间(或无穷度量空间)由满足强三角不等式的相异性函数定义,即三角形任意边不大于另两边中的较大者。我们证明,在超度量空间中使用视点树进行搜索时,最坏情况下的复杂度等于树的深度。由于实际数据集通常非超度量,我们引入一个投影算子,将任意相异性函数转换为超度量空间,同时保持最近邻关系。进一步地,我们学习该投影算子的近似形式,以高效计算查询点与数据集点之间的超度量距离。在此基础上,我们解决更一般的问题:考虑在q-度量空间中的投影——其中三角形各边的q次幂之和大于其余两边的q次幂之和。由于使用了学习得到的投影近似,整个搜索流程变为近似方法。实验表明,随着q值增大,搜索速度加快但召回率下降。总体而言,q-度量与无穷度量空间中的搜索性能可与现有方法媲美。
原文摘要 · Abstract (English)
An ultrametric space or infinity-metric space is defined by a dissimilarity function that satisfies a strong triangle inequality in which every side of a triangle is not larger than the larger of the other two. We show that search in ultrametric spaces with a vantage point tree has worst-case complexity equal to the depth of the tree. Since datasets of interest are not ultrametric in general, we employ a projection operator that transforms an arbitrary dissimilarity function into an ultrametric space while preserving nearest neighbors. We further learn an approximation of this projection operator to efficiently compute ultrametric distances between query points and points in the dataset. We proceed to solve a more general problem in which we consider projections in $q$-metric spaces -- in which triangle sides raised to the power of $q$ are smaller than the sum of the $q$-powers of the other two. Notice that the use of learned approximations of projected $q$-metric distances renders the search pipeline approximate. We show in experiments that increasing values of $q$ result in faster search but lower recall. Overall, search in q-metric and infinity metric spaces is competitive with existing search methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。