用浮点8位量化+降维，实现8倍压缩且损失极小。
Optimization of embeddings storage for RAG systems using quantization and dimensionality reduction techniques
- 采用float8量化与PCA降维协同优化
- 组合方案达8倍压缩，性能损失小于0.3%
- 提供可视化工具帮助选最优配置
检索增强生成系统依赖外部知识库的高维向量嵌入，通常以float32精度存储，带来显著内存压力。为解决此问题，我们在MTEB基准上系统评估两种互补优化策略：量化（比较float16、int8、二值化及float8）和降维（包括PCA、核PCA、UMAP、随机投影与自编码器）。结果表明，float8量化在4倍压缩下性能下降不足0.3%，显著优于同压缩比下的int8；PCA在降维中表现最佳。关键发现：适度降维（保留50%维度）与float8结合可实现8倍总压缩，性能损失远低于仅用int8（仅4倍压缩）。为此，我们提出基于性能-存储权衡空间可视化的实用方法，帮助用户在特定内存约束下找到最优配置。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation enhances language models by retrieving relevant information from external knowledge bases, relying on high-dimensional vector embeddings typically stored in float32 precision. However, storing these embeddings at scale presents significant memory challenges. To address this issue, we systematically investigate on MTEB benchmark two complementary optimization strategies: quantization, evaluating standard formats (float16, int8, binary) and low-bit floating-point types (float8), and dimensionality reduction, assessing methods like PCA, Kernel PCA, UMAP, Random Projections and Autoencoders. Our results show that float8 quantization achieves a 4x storage reduction with minimal performance degradation (<0.3%), significantly outperforming int8 quantization at the same compression level, being simpler to implement. PCA emerges as the most effective dimensionality reduction technique. Crucially, combining moderate PCA (e.g., retaining 50% dimensions) with float8 quantization offers an excellent trade-off, achieving 8x total compression with less performance impact than using int8 alone (which provides only 4x compression). To facilitate practical application, we propose a methodology based on visualizing the performance-storage trade-off space to identify the optimal configuration that maximizes performance within their specific memory constraints.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。