提出注意力引导聚类压缩多向量索引，提升跨模态检索效率。
Multi-Vector Index Compression in Any Modality
- 用注意力机制选关键语义区域作为聚类中心，动态压缩文档表示。
- 在文本、视觉文档和视频数据集上，性能接近全量索引且节省存储。
- 适合处理图像、视频等高维长序列的高效检索场景。
我们研究任意模态下晚交互式多向量检索的高效方法。晚交互已成为文本、图像、视觉文档和视频信息检索的主流范式，但其计算与存储成本随文档长度线性增长，对图像、视频和音频密集型语料库代价高昂。为此，我们探索在固定向量预算下不依赖查询的多向量文档表示压缩方法。提出四种索引压缩策略：序列重缩放、记忆令牌、分层池化，以及一种新颖的注意力引导聚类（AGC）。AGC利用注意力机制识别文档中最语义显著的区域作为聚类中心，并对令牌聚合进行加权。在涵盖文本（BEIR）、视觉文档（ViDoRe）和视频（MSR-VTT, MultiVENT 2.0）的检索任务上评估表明，注意力引导聚类持续优于其他参数化压缩方法（如序列重缩放和记忆令牌），相比非参数化分层聚类具有更大索引尺寸灵活性，且性能达到或超越完整未压缩索引。源代码已开源：github.com/hanxiangqin/omni-col-press。
原文摘要 · Abstract (English)
We study efficient multi-vector retrieval for late interaction in any modality. Late interaction has emerged as a dominant paradigm for information retrieval in text, images, visual documents, and videos, but its computation and storage costs grow linearly with document length, making it costly for image-, video-, and audio-rich corpora. To address this limitation, we explore query-agnostic methods for compressing multi-vector document representations under a constant vector budget. We introduce four approaches for index compression: sequence resizing, memory tokens, hierarchical pooling, and a novel attention-guided clustering (AGC). AGC uses an attention-guided mechanism to identify the most semantically salient regions of a document as cluster centroids and to weight token aggregation. Evaluating these methods on retrieval tasks spanning text (BEIR), visual-document (ViDoRe), and video (MSR-VTT, MultiVENT 2.0), we show that attention-guided clustering consistently outperforms other parameterized compression methods (sequence resizing and memory tokens), provides greater flexibility in index size than non-parametric hierarchical clustering, and achieves competitive or improved performance compared to a full, uncompressed index. The source code is available at: github.com/hanxiangqin/omni-col-press.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。