让检索模型按查询动态选关键文字,省存储又提准度。
Query-Aware Token Budgeting for Efficient Late-Interaction Visual Document Retrieval

- 先用压缩索引快速找候选页,再按查询需求选最优文本块。
- 贪心策略找回98.39%原始精度,比固定选前k块提升显著。
- 适合需要高精度与低延迟的文档检索系统开发者。
晚期交互式视觉文档检索器通过每页保留大量标记嵌入来保存细粒度页面证据,但由此带来的存储和查询时交互成本使大规模部署昂贵。在索引前对文档标记进行池化是一种自然解决方案,但静态池化必须在查询前决定保留哪些视觉证据。本文提出一种替代方案:先通过高度压缩的热路径索引生成候选,再对候选页的原始标记集进行查询感知的标记预算分配。我们将这一阶段二的选择建模为预算约束的最大相似度覆盖问题,证明其截断版本具有单调子模性,并比较了仅覆盖、聚类引导、逐标记和边际增益等策略。在十项 ViDoRe 任务上,使用 ColModernVBERT 时,直接静态池化将五阶宏 NDCG 从无压缩时的 0.6309 降至 0.4738(池化因子 32 倍)。在相同候选生成方式与等效池化因子 8 的重排序预算下,标记 top-k 恢复了 93.93% 的全标记得分,而贪婪边际增益选择恢复了 98.39%。留一数据集外与持留评估均显示,贪婪策略在所有数据集上优于 top-k。延迟分析揭示两个有效运行点:top-k 适用于交互式检索,朴素贪婪实现则作为质量上限。这些结果表明,晚期交互检索受益于查询感知分配,而非仅依赖查询无关的池化。
原文摘要 · Abstract (English)
Late-interaction visual document retrievers preserve fine-grained page evidence by storing many token embeddings per page, but the resulting storage and query-time interaction costs make large-scale deployment expensive. Pooling document tokens before indexing offers a natural remedy, yet static pooling must decide which visual evidence to preserve before the query is known. We study an alternative: a heavily compressed hot-path index generates candidates, after which query-aware token budgeting operates on the original token sets of the shortlisted pages. We formulate this stage-two selection as a budgeted MaxSim coverage problem, show that a clipped version is monotone submodular, and compare coverage-only, cluster-guided, token-wise, and marginal-gain policies. On ten ViDoRe tasks with ColModernVBERT, direct static pooling reduces macro normalized discounted cumulative gain at rank five from 0.6309 without compression to 0.4738 at a thirty-two-fold pool factor. Under the same candidate-generation regime and a pool-factor-eight-equivalent reranking budget, token top-k recovers 93.93 percent of the full-token score, while greedy marginal-gain selection recovers 98.39 percent. Held-out and leave-one-dataset-out evaluations yield positive greedy improvements over token top-k on every dataset. The latency analysis reveals two useful operating points: token top-k for interactive retrieval and the naive greedy implementation as a quality upper envelope. Together, these results show that late-interaction visual retrieval benefits from query-aware allocation rather than query-agnostic pooling alone.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。