arXiv:2412.11023cs.CV2024-12AAAI被引 81

用Mamba持续传递视频上下文信息,提升目标跟踪鲁棒性

Exploring Enhanced Contextual Information for Video-Level Object Tracking

  • 用Mamba层存储历史上下文,跨注意力层融合到当前特征
  • 在LaSOT上达76.6% AUC,GOT-10k上达80.0% AO
  • 适合需要长时依赖建模的视频跟踪任务

视频级上下文信息对视觉目标跟踪愈发重要。然而现有方法通常仅用少量标记传递上下文,易造成信息丢失,难以充分捕捉上下文。为此,我们提出一种名为MCITrack的新框架,利用Mamba的隐藏状态持续记录并传输视频流中的广泛上下文信息,从而实现更鲁棒的目标跟踪。其核心是上下文信息融合模块,包含Mamba层和交叉注意力层:前者存储历史上下文,后者将该信息融入每个骨干块的当前视觉特征中。该模块通过与骨干网络深度集成,增强模型在多层级上捕捉和利用上下文的能力。实验表明,MCITrack在多个基准测试中表现优异,例如在LaSOT上达到76.6% AUC,在GOT-10k上达到80.0% AO,刷新了当前最佳性能。代码与模型已公开于https://github.com/kangben258/MCITrack。

原文摘要 · Abstract (English)

Contextual information at the video level has become increasingly crucial for visual object tracking. However, existing methods typically use only a few tokens to convey this information, which can lead to information loss and limit their ability to fully capture the context. To address this issue, we propose a new video-level visual object tracking framework called MCITrack. It leverages Mamba's hidden states to continuously record and transmit extensive contextual information throughout the video stream, resulting in more robust object tracking. The core component of MCITrack is the Contextual Information Fusion module, which consists of the mamba layer and the cross-attention layer. The mamba layer stores historical contextual information, while the cross-attention layer integrates this information into the current visual features of each backbone block. This module enhances the model's ability to capture and utilize contextual information at multiple levels through deep integration with the backbone. Experiments demonstrate that MCITrack achieves competitive performance across numerous benchmarks. For instance, it gets 76.6% AUC on LaSOT and 80.0% AO on GOT-10k, establishing a new state-of-the-art performance. Code and models are available at https://github.com/kangben258/MCITrack.

视频跟踪上下文建模Mamba

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。