用主成分分析静态降维，提升稠密检索效率
Static Pruning in Dense Retrieval using Matrix Decomposition
- 通过主成分分析实现无需查询的静态嵌入降维
- 降维超50%且NDCG@10最多仅下降5%
- 适合追求高效检索的系统部署场景
在稠密检索时代，文档索引与检索主要依赖将文本转换为嵌入向量的编码模型。检索效率与文档数量及嵌入维度直接相关。近期研究显示可在不损失甚至提升检索效果的前提下压缩嵌入大小，但这些方法依赖查询，无法离线处理，需额外计算，影响效率。本文提出一种基于主成分分析的静态降维方法，可离线执行且与查询无关，显著提升稠密检索效率，对系统效果影响极小。实验表明，该方法使文档表示维度降低超过50%，在不同稠密检索模型上NDCG@10最高仅下降5%。
原文摘要 · Abstract (English)
In the era of dense retrieval, document indexing and retrieval is largely based on encoding models that transform text documents into embeddings. The efficiency of retrieval is directly proportional to the number of documents and the size of the embeddings. Recent studies have shown that it is possible to reduce embedding size without sacrificing - and in some cases improving - the retrieval effectiveness. However, the methods introduced by these studies are query-dependent, so they can't be applied offline and require additional computations during query processing, thus negatively impacting the retrieval efficiency. In this paper, we present a novel static pruning method for reducing the dimensionality of embeddings using Principal Components Analysis. This approach is query-independent and can be executed offline, leading to a significant boost in dense retrieval efficiency with a negligible impact on the system effectiveness. Our experiments show that our proposed method reduces the dimensionality of document representations by over 50% with up to a 5% reduction in NDCG@10, for different dense retrieval models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。