arXiv:2604.01974cs.CV2026-04

让追踪器能听懂人话,实时响应指令,实现人机协同追踪。

Interactive Tracking: A Human-in-the-Loop Paradigm with Memory-Augmented Adaptation

  • 用自然语言指令动态调整追踪行为,引入人机交互新范式。
  • 构建首个大规模交互追踪基准InteractTrack,含150段带标注视频和语音指令。
  • 提出IMAT模型,通过动态记忆学习用户反馈,提升适应性与协作能力。

现有视觉追踪器多为非交互式、一次性运行,难以适应需人工介入的真实场景。为此,本文提出交互追踪(Interactive Tracking)新范式,允许用户随时通过自然语言指令引导追踪过程。主要贡献包括:首先,构建首个大规模交互追踪基准InteractTrack,包含150个视频,配有密集边界框标注及时间戳语言指令;其次,提出全面评估协议,对25种代表性追踪器进行评测,发现顶尖方法在交互场景中表现不佳,传统基准性能无法迁移;第三,提出交互记忆增强追踪(IMAT),采用动态记忆机制学习用户反馈并自适应更新追踪策略。该基准、评估协议与基线为发展更智能、可适应、可协作的追踪系统奠定基础,弥合自动化感知与人类引导之间的差距。完整数据集、结果与分析详见https://github.com/NorahGreen/InteractTrack.git。

原文摘要 · Abstract (English)

Existing visual trackers mainly operate in a non-interactive, fire-and-forget manner, making them impractical for real-world scenarios that require human-in-the-loop adaptation. To overcome this limitation, we introduce Interactive Tracking, a new paradigm that allows users to guide the tracker at any time using natural language commands. To support research in this direction, we make three main contributions. First, we present InteractTrack, the first large-scale benchmark for interactive tracking, containing 150 videos with dense bounding box annotations and timestamped language instructions. Second, we propose a comprehensive evaluation protocol and evaluate 25 representative trackers, showing that state-of-the-art methods fail in interactive scenarios; strong performance on conventional benchmarks does not transfer. Third, we introduce Interactive Memory-Augmented Tracking (IMAT), a new baseline that employs a dynamic memory mechanism to learn from user feedback and update tracking behavior accordingly. Our benchmark, protocol, and baseline establish a foundation for developing more intelligent, adaptive, and collaborative tracking systems, bridging the gap between automated perception and human guidance. The full benchmark, tracking results, and analysis are available at https://github.com/NorahGreen/InteractTrack.git.

交互追踪人机协同动态记忆自然语言

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。