用一个可学习的标记精挑细选视觉特征，提升图文模型检索性能。
ReToken: One Token to Improve Vision-Language Models for Visual Retrieval
- 引入单个可学习嵌入，从视觉缓存中挑选与查询相关的稀疏特征。
- 在图像和视频检索任务上显著提升效果，最高增益达13.4分。
- 轻量设计支持单卡训练与长视频推理，适合实际部署。
长视觉上下文给图文模型带来挑战：干扰项增多时性能下降，且全量处理所有视觉标记受显存限制难以实现。我们提出 ReToken，一种作为显式检索目标的单一可学习嵌入，能从预填充的视觉键值缓存中筛选出与查询相关的稀疏视觉标记。仅在小规模图像问答数据集上训练，ReToken 在多个图像与视频基准上均表现稳定提升：在 Visual Haystacks 上使 Qwen3VL-8B 提升 13.4 分，InternVL3.5 提升 12.4 分（相对提升超 20%）；在 LVBench 上零样本迁移到长视频任务，使 Qwen3VL-8B 提升 8.0 分。得益于其轻量设计，训练与长视频推理均可在单张 H100 上完成。代码已开源。
原文摘要 · Abstract (English)
Long visual context poses a challenge for vision-language models: performance degrades as the number of distractors grows, and processing all tokens at once is computationally infeasible under GPU memory constraints. We present ReToken, a single learnable embedding trained as an explicit retrieval target that selects a sparse set of query-relevant visual tokens from a pre-filled visual KV cache. Trained on only a small image-QA dataset, ReToken yields consistent gains across image and video benchmarks: on Visual Haystacks it improves Qwen3VL-8B by 13.4 points and InternVL3.5 by 12.4 points (>20% relative), and on LVBench it transfers zero-shot to long video for an 8.0-point gain with Qwen3VL-8B. Thanks to its lightweight design, both training and long-video inference fit on a single H100. Code is available at: https://github.com/avaxiao/ReToken
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。