提出RAE方法，让降维后向量的近邻关系不变，提升检索效率。
RAE: A Neural Network Dimensionality Reduction Method for Nearest Neighbors Preservation in Vector Search
- 用正则化自编码器控制降维时向量大小变化，保持近邻结构。
- 在多个数据集上，比传统方法提升近邻召回率，且推理快。
- 适合需要高精度检索的推荐系统与生成任务场景。
随着高维嵌入向量在检索增强生成和推荐系统等任务中广泛应用，由于主成分分析（PCA）和UMAP等主流降维方法难以保持向量间的最近邻（k-NN）关系，其在加速检索过程中的应用受限。本文提出一种基于神经网络优化能力与瑞利商有界性的正则化自编码器（RAE），通过正则项约束网络参数变化，调节奇异值以控制降维过程中嵌入向量的幅度变化，从而保留k-NN关系。我们提供了严格的数学分析，证明正则化可建立变换后向量范数畸变率的上界，为k-NN保真提供可证明保证。在少量训练开销下，RAE相比现有降维方法实现了更高的k-NN召回率，同时保持快速检索效率。
原文摘要 · Abstract (English)
While high-dimensional embedding vectors are being increasingly employed in various tasks like Retrieval-Augmented Generation and Recommendation Systems, popular dimensionality reduction (DR) methods such as PCA and UMAP have rarely been adopted for accelerating the retrieval process due to their inability of preserving the nearest neighbor (NN) relationship among vectors. Empowered by neural networks' optimization capability and the bounding effect of Rayleigh quotient, we propose a Regularized Auto-Encoder (RAE) for k-NN preserving dimensionality reduction. RAE constrains the network parameter variation through regularization terms, adjusting singular values to control embedding magnitude changes during reduction, thus preserving k-NN relationships. We provide a rigorous mathematical analysis demonstrating that regularization establishes an upper bound on the norm distortion rate of transformed vectors, thereby offering provable guarantees for k-NN preservation. With modest training overhead, RAE achieves superior k-NN recall compared to existing DR approaches while maintaining fast retrieval efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。