用推理生成动态语言描述，提升长时视觉语言跟踪精度
ReasoningTrack: Chain-of-Thought Reasoning for Long-term Vision-Language Tracking
- 基于大模型推理生成实时语言描述，动态适配目标变化
- 在TNLLT数据集上相较基线提升12.3% mAP，长时跟踪效果显著
- 适合需要精准目标理解的自动驾驶、机器人视觉场景
视觉语言跟踪近年来受到广泛关注，因文本信息可有效缓解目标指定僵化与不准确问题。现有方法或直接融合固定语言特征，或简单使用注意力机制，性能受限。近期研究尝试通过文本生成适应目标变化，但缺乏对模型推理过程的解释性，未能充分挖掘大模型优势。为此，本文提出基于预训练多模态模型Qwen2.5-VL的新型推理式视觉语言跟踪框架ReasoningTrack，结合监督微调（SFT）与强化学习GRPO优化推理与语言生成。将更新后的语言描述与视觉特征共同输入统一追踪主干网络，再通过追踪头预测目标位置。此外，构建大规模长时视觉语言跟踪基准数据集TNLLT，包含200个视频序列，重新训练并评估20个基线视觉追踪器，为该任务奠定坚实基础。在多个视觉语言跟踪基准数据集上的大量实验验证了所提推理式自然语言生成策略的有效性。
原文摘要 · Abstract (English)
Vision-language tracking has received increasing attention in recent years, as textual information can effectively address the inflexibility and inaccuracy associated with specifying the target object to be tracked. Existing works either directly fuse the fixed language with vision features or simply modify using attention, however, their performance is still limited. Recently, some researchers have explored using text generation to adapt to the variations in the target during tracking, however, these works fail to provide insights into the model's reasoning process and do not fully leverage the advantages of large models, which further limits their overall performance. To address the aforementioned issues, this paper proposes a novel reasoning-based vision-language tracking framework, named ReasoningTrack, based on a pre-trained vision-language model Qwen2.5-VL. Both SFT (Supervised Fine-Tuning) and reinforcement learning GRPO are used for the optimization of reasoning and language generation. We embed the updated language descriptions and feed them into a unified tracking backbone network together with vision features. Then, we adopt a tracking head to predict the specific location of the target object. In addition, we propose a large-scale long-term vision-language tracking benchmark dataset, termed TNLLT, which contains 200 video sequences. 20 baseline visual trackers are re-trained and evaluated on this dataset, which builds a solid foundation for the vision-language visual tracking task. Extensive experiments on multiple vision-language tracking benchmark datasets fully validated the effectiveness of our proposed reasoning-based natural language generation strategy. The source code of this paper will be released on https://github.com/Event-AHU/Open_VLTrack
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。