用预训练模型自动生成视觉提示,提升目标追踪的区分能力
Improving Visual Object Tracking through Visual Prompting
- 基于CLIP自动在线生成并优化视觉提示,实现动态特征增强
- 在多个基准上显著降低干扰物影响,提升追踪准确率
- 适合需要高鲁棒性追踪的视觉系统开发者参考
学习能区分目标与背景干扰物的判别性模型是通用目标追踪(GOT)的关键。现有追踪器在面对动态干扰时判别能力有限,难以实现有效适应。为此,本文提出一种新型视觉提示机制PiVOT,利用预训练基础模型CLIP在线自动生成并精炼视觉提示,通过对比引导抑制干扰物。PiVOT设计了提示初始化机制,生成聚焦潜在目标位置的初始提示;随后利用CLIP根据候选对象与参考模板之间的外观相似性进行提示优化,使提示更精准地突出目标区域并减少无关信息。结合该提示机制,追踪器可生成由提示引导的实例感知特征图,并在追踪过程中持续、自动更新,有效抑制干扰。大量实验表明,该方法在多个基准上均能有效减少干扰物影响,提升追踪性能。
原文摘要 · Abstract (English)
Learning a discriminative model that distinguishes the specified target from surrounding distractors across frames is essential for generic object tracking (GOT). Dynamic adaptation of target representation against distractors remains challenging because prevailing trackers exhibit limited discriminative capability. To address this issue, we present a new visual prompting mechanism for generic object tracking, termed PiVOT. PiVOT introduces mechanisms that leverage the pretrained foundation model (CLIP) to automatically generate and refine visual prompts online, thereby enabling the tracker to suppress distractors through contrastive guidance. To transfer contrastive knowledge from the foundation model to the tracker, PiVOT automatically propagates this knowledge online and dynamically generates and updates visual prompts. Specifically, it proposes a prompt initialization mechanism that produces an initial visual prompt highlighting potential target locations. The foundation model is then used to refine the prompt based on appearance similarities between candidate objects and reference templates across potential targets. After refinement, the visual prompt better highlights potential target locations and reduces irrelevant prompt information. With the proposed prompting mechanism, the tracker can generate instance-aware feature maps guided by the visual prompts, which are incrementally and automatically updated during tracking, thereby effectively suppressing distractors. Extensive experiments across multiple benchmarks indicate that PiVOT, with the proposed prompting mechanism, can suppress distracting objects and improve tracking performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。