一种可同时支持多级压缩的文档检索训练方法，节省存储且灵活可调。
Spatial Matryoshka Training for Multi-Granularity Visual Document Retrieval
- 通过空间分层池化生成多粒度压缩层级，单次编码输出所有层级
- 在大幅压缩下仍保持接近全分辨率的检索性能
- 轻量适配即可提升现有模型，适合资源受限场景
多模态晚期交互检索器通过将每页文档表示为块嵌入并在令牌级别匹配，在视觉丰富的文档中实现了强大的检索能力。然而，这种方法带来高昂的存储成本。现有压缩方法通常在索引时固定单一压缩级别，限制了灵活性。本文提出ColSNAP（空间嵌套平均池化），一种从骨干网络的块网格直接生成嵌套压缩层级的训练方法。通过空间池化块嵌入为逐渐粗化的层级，并同时训练所有层级，单个模型无需架构改动即可支持多级压缩。关键在于，一次编码即可获得所有层级，使准确率与存储的权衡可在索引时根据可用存储预算动态配置，而非训练时固定。实验表明，使用ColSNAP训练的模型在大幅压缩下仍保持接近全分辨率的检索性能，且在多个晚期交互骨干模型间具有良好迁移性，其大部分性能提升来自对预训练检索器施加的轻量适配阶段。
原文摘要 · Abstract (English)
Multi-modal late-interaction retrievers achieve strong retrieval on visually rich documents by representing each page as per patch embeddings and matching at the token level. However, this approach incurs high storage costs. Existing compression methods typically fix a single compression level at indexing time, limiting flexibility. We present ColSNAP (Spatial Nested Average Pooling)1, a training method that generates a nested hierarchy of compression levels directly from a backbone's patch grid. By spatially pooling patch embeddings into pro- gressively coarser tiers and training all tiers simultaneously, a single model learns to support retrieval at multiple compression levels without architectural changes. Crucially, a single encoding pass yields every tier, enabling the accuracy-storage trade-off to be configured at indexing time to match avail- able storage budgets, rather than being fixed during training. We demonstrate that models trained using ColSNAP maintain near full-resolution retrieval performance under substantial compression and that ColSNAP transfers effectively across multiple late-interaction backbones, and achieves most of its improvements via a lightweight adaptation stage applied to a pre-trained retriever.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。