动态调整视觉分辨率，高效定位第一视角中的小物体。
Dynamic Resolution Routing for Efficient Egocentric Grounding
- 在像素层面动态路由，只对关键区域保持高分辨率。
- 减少67%视觉标记，仍保留86.4%的全分辨率性能。
- 适合需要高效处理小物体的第一视角应用。
第一视角视觉定位需高分辨率输入以精确定位小物体，但多模态大模型在此领域受限于视觉标记处理的高昂开销。现有基于标记缩减的高效策略在选择以物体为中心的空间证据时不可靠。为此，我们提出SmartRes框架，通过像素空间的动态分辨率路由实现效率优化。SmartRes先编码低分辨率图像获取全局上下文，再用轻量级路由器激活物体相关区域的高分辨率块，并构建保持顺序的视觉序列。为应对前景-背景严重失衡带来的路由不稳问题，引入边际正则化路由目标，增强前景与背景特征分离，提升前景召回率。在Ego4D和EgoIntention数据集上的实验表明，SmartRes可将视觉标记减少高达67%，同时保留86.4%的全分辨率性能，推理速度比当前最优标记缩减方法快1.66倍且精度更高。对小物体定位的优异表现验证了该方法在第一视角场景中的有效性。代码将公开。
原文摘要 · Abstract (English)
Egocentric visual grounding requires high-resolution inputs to localize small objects. However, scaling Multimodal Large Language Models to this domain is constrained by the excessive cost of visual token processing. We identify that current efficient strategies based on token reduction are unreliable for selecting object-centric spatial evidence. To overcome this, we propose SmartRes, a framework that performs efficiency optimization in the pixel space via dynamic resolution routing. SmartRes first encodes a low-resolution view for global context and uses a lightweight router to activate high-resolution patches in object-centric regions and constructs an order-preserving visual sequence. To further enable robust routing under severe foreground-background imbalance, we introduce a margin-regularized routing objective that increases foreground-background logit separation and improves foreground recall. Experiments on Ego4D and EgoIntention show that SmartRes reduces visual tokens by up to 67% while retaining 86.4% of full-resolution performance, and achieves up to 1.66X faster inference than state-of-the-art token reduction methods with higher accuracy. Furthermore, strong performance on small object grounding indicates the effectiveness of SmartRes towards egocentric applications. Code will be publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。