用异常模式检测实现多查询向量检索,提升复杂任务的匹配精度
Retrieval with Multiple Query Vectors through Anomalous Pattern Detection

- 通过检测多个查询向量中的异常维度,定位关键特征
- 在图像、文本和表格数据上,查询集越大性能越好,8个时提升最明显
- 适合需要多角度推理的复杂检索场景,如跨模态搜索
经典向量检索通常以单个查询嵌入向量为输入,从向量数据库中检索最相似的向量。然而,复杂推理与检索任务往往需要多个查询向量而非单一向量。本文提出一种新方法,同时处理多个查询向量,并借助异常模式检测思想,识别查询集合Q中显著偏离其他维度的子集。随后,在向量数据库中扫描,找出在这些异常维度上同样表现出异常的向量,并将其作为检索结果返回。我们在两个图像数据集、一个文本数据集和一个表格数据集上验证该方法。总体结果显示,多数数据集上更大的查询集能带来更好的检索性能,尤其在从1个增至8个查询时提升最为显著,之后增益逐渐减小。
原文摘要 · Abstract (English)
A classical vector retrieval problem typically considers a \emph{single} query embedding vector as input and retrieves the most similar embedding vectors from a vector database. However, complex reasoning and retrieval tasks frequently require \emph{multiple query vectors}, rather than a single one. In this work, we propose a retrieval method that considers multiple query vectors simultaneously and retrieves the most relevant vectors from the database using concepts from anomalous pattern detection. Specifically, our approach leverages a set of query vectors $Q$ (with $|Q|\geq 1$), and identifies the subset of vector dimensions within $Q$ that standout (anomalous) from the rest of dimensions. Next, we scan the vector database to retrieve the set of vectors that are also anomalous across the previously identified vector dimensions and return them as our retrieved set of vectors. We validate our approach on two image datasets, a text dataset, and a tabular dataset. Overall, we observe that, across most datasets, larger query sets lead to improved retrieval performance. The improvement is most pronounced when increasing the query sets from 1 to 8, while the gains become smaller beyond that.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。