提出可恢复的视觉标记路由机制，提升模型在低资源下的视觉定位能力。
Reroute, Don't Remove: Recoverable Visual Token Routing for Vision-Language Models
- 用可逆路由替代传统删除，让被跳过的标记可在后续阶段重新参与计算
- 在极端降采样下仍保持精准视觉定位，通用视觉问答性能不变
- 无需训练、即插即用，适配多种主流视觉语言模型
视觉语言模型将图像投影为数百至数千个视觉标记，导致解码器推理在注意力计算和键值缓存内存方面开销巨大。现有视觉标记压缩方法多采用评分后删除的策略：对标记排序，保留紧凑子集并永久丢弃其余部分。我们发现这种不可逆操作脆弱，因为视觉标记的重要性随解码器深度变化；某些阶段排名较低的标记在后期层中可能变得关键，尤其对需要精确定位的查询。为此，我们提出Reroute——一种无需训练的即插即用模块，将删除改为可恢复的路由。在每个路由阶段，选定的视觉标记进入解码器块，而被推迟的标记则跳过该阶段，并在下一决策阶段重新进入候选池。Reroute复用现有的注意力得分排序规则与分阶段调度策略，保持所增强压缩方法的理论计算量（TFLOPs）和键值缓存预算类别。在LLaVA-1.5与Qwen骨干网络上的FastV、PDrop及Nüwa变体上，该方法在激进的标记压缩下显著提升定位性能，同时维持通用视觉问答表现。结果表明，视觉标记压缩不应仅被视为不可逆剪枝，更应看作可恢复的路由过程。代码见：https://github.com/elmma/mllm-reroute/
原文摘要 · Abstract (English)
Vision-language models (VLMs) project images into hundreds to thousands of visual tokens, making decoder inference expensive in both attention computation and KV-cache memory. Existing visual-token reduction methods largely follow a rank-and-remove paradigm: they score visual tokens, keep a compact subset, and permanently discard the rest. We show that this irreversible action is fragile because visual-token importance changes across decoder depth; tokens ranked low at one stage may become relevant in later layers, especially for grounding-sensitive queries. We propose Reroute, a training-free plug-in that replaces removal with recoverable routing. At each routing stage, selected vision tokens pass through decoder blocks, while deferred tokens bypass the stage and re-enter the candidate pool at the next routing decision. Reroute reuses existing attention-score ranking rules and stage-wise schedules, preserving the theoretical TFLOPs and KV-cache budget class of the pruning method it augments. Across FastV, PDrop, and Nüwa variants on LLaVA-1.5 and Qwen backbones, reroute improves grounding under aggressive token reduction while maintaining general VQA performance. These results suggest that VLM token reduction should not be viewed only as irreversible pruning, but also as recoverable routing. The code can be found here: https://github.com/elmma/mllm-reroute/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。