arXiv:2602.00570cs.CV2026-02IJCV被引 2

用生成模型提升模糊模板的语义,让视觉跟踪更准更快。

GLAD: Generative Language-Assisted Visual Tracking for Low-Semantic Templates

  • 用扩散模型生成语义丰富的模板图像,融合文本与视觉信息
  • 在多个基准上达到新最好性能,推理速度显著提升
  • 适合处理低质量、模糊或语义不清的视觉跟踪场景

视觉-语言跟踪在诸多场景中受到关注,需同时处理视觉与语言信息以定位视频中的目标。尽管该任务日益重要,当前方法仍处于初期阶段。现有追踪器通常采用Transformer架构融合模板、搜索区域与文本特征,但低语义图像(如模糊、低分辨率)会削弱跨模态理解能力。语言辅助虽被用于缓解此问题,但文本与视觉特征间存在差距,直接拼接融合效果有限。为此,我们提出开创性的生成式语言辅助跟踪模型GLAD,利用扩散模型生成多模态融合的文本描述与模板图像,增强语言与图像的兼容性,并提升模板的语义信息。该方法可有效恢复模糊和语义模糊的模板图像,改善融合效果。实验表明,本方法在多个基准上达到新最佳性能,且推理速度优异。代码与模型将开源:https://github.com/Confetti-lxy/GLAD

原文摘要 · Abstract (English)

Vision-language tracking has gained increasing attention in many scenarios. This task simultaneously deals with visual and linguistic information to localize objects in videos. Despite its growing utility, the development of vision-language tracking methods remains in its early stage. Current vision-language trackers usually employ Transformer architectures for interactive integration of template, search, and text features. However, persistent challenges about low-semantic images including prevalent image blurriness, low resolution and so on, may compromise model performance through degraded cross-modal understanding. To solve this problem, language assistance is usually used to deal with the obstacles posed by low-semantic images. However, due to the existing gap between current textual and visual features, direct concatenation and fusion of these features may have limited effectiveness. To address these challenges, we introduce a pioneering Generative Language-AssisteD tracking model, GLAD, which utilizes diffusion models for the generative multi-modal fusion of text description and template image to bolster compatibility between language and image and enhance template image semantic information. Our approach demonstrates notable improvements over the existing fusion paradigms. Blurry and semantically ambiguous template images can be restored to improve multi-modal features in the generative fusion paradigm. Experiments show that our method establishes a new state-of-the-art on multiple benchmarks and achieves an impressive inference speed. The code and models will be released at: https://github.com/Confetti-lxy/GLAD

视觉跟踪生成模型多模态融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。