LEANN通过动态计算嵌入并压缩索引，将向量搜索存储量降低至原数据的5%
LEANN: A Low-Storage Vector Index
- 不存储嵌入，运行时动态计算，减少冗余存储
- 在真实数据集上索引体积缩小最高50倍，精度保持顶尖水平
- 适合资源受限设备或大规模数据部署，支持高效更新
基于嵌入的向量搜索支撑推荐和检索增强生成（RAG）等重要应用，依赖向量索引实现高效搜索。但传统索引需存储高维嵌入和大量元数据，总大小可达原始数据的数倍（如文本块）。这使得在个人设备或大规模数据上部署向量搜索变得困难甚至不可行。为此，我们提出LEANN，一种存储高效的向量索引：它不存储嵌入，而是按需动态计算；同时压缩最先进的邻近图索引，在保持搜索精度的前提下大幅减小存储。LEANN在仅使用原数据约5%存储空间的情况下，实现高质量向量搜索，并支持高效索引构建与更新。在真实世界基准测试中，相比传统索引，其索引大小最多减少50倍，同时维持最先进精度和与RAG应用相匹配的延迟。
原文摘要 · Abstract (English)
Embedding-based vector search underpins many important applications, such as recommendation and retrieval-augmented generation (RAG). It relies on vector indices to enable efficient search. However, these indices require storing high-dimensional embeddings and large index metadata, whose total size can be several times larger than the original data (e.g., text chunks). Such high storage overhead makes it difficult, or even impractical, to deploy vector search on personal devices or large-scale datasets. To tackle this problem, we propose LEANN, a storage-efficient index for vector search that recomputes embeddings on the fly instead of storing them, and compresses state-of-the-art proximity graph indices while preserving search accuracy. LEANN delivers high-quality vector search while using only a fraction of the storage (e.g., 5% of the original data) and supporting storage-efficient index construction and updates. On real-world benchmarks, LEANN reduces index size by up to 50x compared with conventional indices, while maintaining SOTA accuracy and comparable latency for RAG applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。