用视觉提示实现跨图像全局追踪，抗遮挡更稳定。
VPTracker: Global Vision-Language Tracking via Visual Prompt
- 基于多模态大模型的全局搜索+位置感知提示
- 在复杂场景下追踪准确率提升显著，漂移减少
- 适合需要强语义理解的视觉追踪任务
视觉-语言追踪旨在持续定位由视觉模板和语言描述定义的目标。现有方法通常局限于局部搜索，在视角变化、遮挡和快速运动下容易失败。本文提出首个基于多模态大模型（VPTracker）的全局追踪框架，利用其强大的语义推理能力在整幅图像中定位目标。尽管全局搜索提升了鲁棒性并减少漂移，但也易受视觉或语义相似对象干扰。为此，我们设计了一种位置感知的视觉提示机制，将前一帧目标位置作为空间先验构建区域级提示，使模型优先进行区域识别，仅在必要时启用全局推理。该设计有效抑制干扰，同时保留全局追踪优势。大量实验表明，该方法显著提升了复杂场景下的追踪稳定性与目标区分能力，为多模态大模型融入视觉追踪开辟新路径。代码已开源。
原文摘要 · Abstract (English)
Vision-Language Tracking aims to continuously localize objects described by a visual template and a language description. Existing methods, however, are typically limited to local search, making them prone to failures under viewpoint changes, occlusions, and rapid target movements. In this work, we introduce the first global tracking framework based on Multimodal Large Language Models (VPTracker), exploiting their powerful semantic reasoning to locate targets across the entire image space. While global search improves robustness and reduces drift, it also introduces distractions from visually or semantically similar objects. To address this, we propose a location-aware visual prompting mechanism that incorporates spatial priors into the MLLM. Specifically, we construct a region-level prompt based on the target's previous location, enabling the model to prioritize region-level recognition and resort to global inference only when necessary. This design retains the advantages of global tracking while effectively suppressing interference from distracting visual content. Extensive experiments show that our approach significantly enhances tracking stability and target disambiguation under challenging scenarios, opening a new avenue for integrating MLLMs into visual tracking. Code is available at https://github.com/jcwang0602/VPTracker.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。