用4比特量化压缩向量嵌入，提升RAG系统存储与检索效率
4bit-Quantization in Vector-Embedding for RAG
- 将嵌入向量从32位浮点数转为4位整数，大幅降低内存占用
- 在不影响检索精度前提下，实现存储量减少87.5%、搜索速度提升
- 适合资源受限环境下部署RAG系统的研究者与开发者
检索增强生成（RAG）是一种有前景的技术，可缓解大语言模型（LLM）的两大缺陷：训练数据导致的信息过时，以及生成虚假内容（幻觉）。RAG通过使用包含相关文档的数据库来解决这些问题，这些文档以高维空间中的嵌入向量形式存储。然而，高维嵌入向量需大量内存存储，尤其在大规模文档库中尤为显著。为此，本文提出采用4比特量化技术存储嵌入向量，将数值精度从32位浮点数降至4位整数，显著减少内存需求。该方法不仅大幅降低向量数据库的存储开销，使RAG系统更适用于资源受限环境，还加快了检索速度，因低精度计算更高效。实验表明，在主流数据集如NQ、TREC-COVID上，4比特量化仍保持接近原始精度的检索性能。代码已开源于https://github.com/taeheej/4bit-Quantization-in-Vector-Embedding-for-RAG。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) is a promising technique that has shown great potential in addressing some of the limitations of large language models (LLMs). LLMs have two major limitations: they can contain outdated information due to their training data, and they can generate factually inaccurate responses, a phenomenon known as hallucinations. RAG aims to mitigate these issues by leveraging a database of relevant documents, which are stored as embedding vectors in a high-dimensional space. However, one of the challenges of using high-dimensional embeddings is that they require a significant amount of memory to store. This can be a major issue, especially when dealing with large databases of documents. To alleviate this problem, we propose the use of 4-bit quantization to store the embedding vectors. This involves reducing the precision of the vectors from 32-bit floating-point numbers to 4-bit integers, which can significantly reduce the memory requirements. Our approach has several benefits. Firstly, it significantly reduces the memory storage requirements of the high-dimensional vector database, making it more feasible to deploy RAG systems in resource-constrained environments. Secondly, it speeds up the searching process, as the reduced precision of the vectors allows for faster computation. Our code is available at https://github.com/taeheej/4bit-Quantization-in-Vector-Embedding-for-RAG
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。