让视频追踪只关注用户指定目标，用自然语言精准指挥跟踪
QTrack: Query-Driven Reasoning for Multi-modal MOT
- 用自然语言查询驱动追踪，仅关注指定目标
- 构建新数据集RMOT26，支持跨序列泛化评估
- 结合视觉语言模型与时序感知奖励，提升追踪一致性
多对象追踪（MOT）传统上需估计视频中所有物体的轨迹，而无法根据语义指令选择性地推理特定目标。本文提出一种查询驱动的追踪范式，将追踪建模为受自然语言查询条件约束的时空推理问题。给定参考帧、视频序列和文本查询，目标是仅定位并追踪查询中指定的目标，同时保持时间连贯性和身份一致性。为此，我们构建了大规模基准RMOT26，包含具象化查询和序列级划分，防止身份泄露并支持鲁棒的泛化评估。进一步提出端到端的视觉-语言模型QTrack，融合多模态推理与面向追踪的定位能力。此外，引入时序感知奖励策略，以结构化奖励促进运动感知推理。大量实验表明该方法在以推理为中心的语言引导追踪任务中有效。代码与数据见https://github.com/gaash-lab/QTrack。
原文摘要 · Abstract (English)
Multi-object tracking (MOT) has traditionally focused on estimating trajectories of all objects in a video, without selectively reasoning about user-specified targets under semantic instructions. In this work, we introduce a query-driven tracking paradigm that formulates tracking as a spatiotemporal reasoning problem conditioned on natural language queries. Given a reference frame, a video sequence, and a textual query, the goal is to localize and track only the target(s) specified in the query while maintaining temporal coherence and identity consistency. To support this setting, we construct RMOT26, a large-scale benchmark with grounded queries and sequence-level splits to prevent identity leakage and enable robust evaluation of generalization. We further present QTrack, an end-to-end vision-language model that integrates multimodal reasoning with tracking-oriented localization. Additionally, we introduce a Temporal Perception-Aware Policy Optimization strategy with structured rewards to encourage motion-aware reasoning. Extensive experiments demonstrate the effectiveness of our approach for reasoning-centric, language-guided tracking. Code and data are available at https://github.com/gaash-lab/QTrack
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。