通过分层压缩提升文档检索效率,存储减32倍,延迟降50%。
Hierarchical Patch Compression for ColPali: Efficient Multi-Vector Document Retrieval with Dynamic Pruning and Quantization
- 用聚类量化将嵌入压缩为1字节索引,存储减32倍。
- 按注意力动态剪枝,计算量减60%,精度损失<2%。
- 支持二进制编码,适合资源受限环境,加速相似度搜索。
多向量文档检索系统如ColPali在复杂查询下表现优异,但因高维片段嵌入和后期交互评分导致存储与计算开销大。本文提出HPC-ColPali分层片段压缩框架,在保持检索精度的同时显著提升效率。方法包括:(1) 使用K-Means量化将片段嵌入压缩为1字节中心点索引,实现最高32×的存储压缩;(2) 基于视觉-语言模型注意力权重动态剪枝,仅保留前p%最显著片段,使后期交互计算减少60%,nDCG@10损失低于2%;(3) 可选地将中心点索引编码为b比特串(b=⌈log₂K⌉),支持快速汉明距离相似性搜索,适用于资源受限场景。在ViDoRe和SEC-Filings数据集上评估,采用HNSW索引时查询延迟降低30–50%,同时保持高检索精度。集成至法律摘要的RAG流水线中,幻觉率下降30%,端到端延迟减半。代码已开源。
原文摘要 · Abstract (English)
Multi-vector document retrieval systems, such as ColPali, excel in fine-grained matching for complex queries but incur significant storage and computational costs due to their reliance on high-dimensional patch embeddings and late-interaction scoring. To address these challenges, we propose HPC-ColPali, a Hierarchical Patch Compression framework that enhances the efficiency of ColPali while preserving its retrieval accuracy. Our approach integrates three innovative techniques: (1) K-Means quantization, which compresses patch embeddings into 1-byte centroid indices, achieving up to 32$\times$ storage reduction; (2) attention-guided dynamic pruning, utilizing Vision-Language Model attention weights to retain only the top-$p\%$ most salient patches, reducing late-interaction computation by up to 60\% with less than 2\% nDCG@10 loss; and (3) optional binary encoding of centroid indices into $b$-bit strings ($b=\lceil\log_2 K\rceil$), enabling rapid Hamming distance-based similarity search for resource-constrained environments. Evaluated on the ViDoRe and SEC-Filings datasets, HPC-ColPali achieves 30--50\% lower query latency under HNSW indexing while maintaining high retrieval precision. When integrated into a Retrieval-Augmented Generation pipeline for legal summarization, it reduces hallucination rates by 30\% and halves end-to-end latency. These advancements establish HPC-ColPali as a scalable and efficient solution for multi-vector document retrieval across diverse applications. Code is available at https://github.com/DngBack/HPC-ColPali.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。