通过自适应分块压缩视觉文档检索向量，大幅降存储且提精度。
Visual Late Chunking: An Empirical Study of Contextual Chunking for Efficient Visual Document Retrieval
- 基于图像块嵌入与位置先验做分层聚类，动态生成上下文相关向量。
- 在24个数据集上减少90%存储量，nDCG@5平均提升9点。
- 适合需高效部署的视觉文档系统，可无缝接入现有模型。
多向量模型因细粒度匹配能力主导视觉文档检索（VDR），但其高存储与计算成本阻碍实际应用。本文提出ColChunk，一种即插即用的多模态晚期分块框架，通过在图像块嵌入上进行分层聚类，并融合二维位置先验，确保空间-语义一致性，实现内容感知的上下文向量表示。该方法在保留全局上下文的同时显著减少向量数量。在24个VDR数据集上的评估表明，ColChunk实现超过90%的存储降低，同时在代表性单向量模型上带来nDCG@5平均9点的提升。该方法为视觉文档系统中检索精度与效率的平衡提供了实用解决方案。
原文摘要 · Abstract (English)
Multi-vector models dominate Visual Document Retrieval (VDR) due to their fine-grained matching capabilities, but their high storage and computational costs present a major barrier to practical deployment. In this paper, we propose ColChunk, a plug-and-play framework that introduces multimodal late chunking to construct efficient, contextualized multi-vectors. Unlike existing pruning or fixed-token approaches, ColChunk employs hierarchical clustering on patch-level embeddings, fused with a 2D position prior to ensure spatial-semantic coherence. This adaptive grouping allows for a content-aware representation that preserves global context while drastically reducing the vector count. Evaluations across 24 VDR datasets demonstrate ColChunk achieves over a 90% reduction in storage requirements while simultaneously delivering a 9-point average improvement in nDCG@5 across representative single-vector models. ColChunk provides a practical solution for balancing retrieval accuracy and efficiency in visual document systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。