arXiv:2511.17967cs.CV2025-11AAAI被引 15

提出CADTrack框架,提升复杂环境下多模态目标跟踪的鲁棒性。

CADTrack: Learning Contextual Aggregation with Deformable Alignment for Robust RGBT Tracking

  • 用状态空间模型实现高效跨模态特征交互,降低计算开销。
  • 通过稀疏门控机制动态激活深层特征,融合互补上下文信息。
  • 结合可变形采样与时序传播,缓解空间错位和定位漂移问题。

RGB-热成像(RGBT)跟踪旨在利用可见光与热红外模态实现全天候鲁棒的目标跟踪。然而,现有方法难以解决模态差异问题,导致特征表示不充分,阻碍跨模态信息传播与融合,显著降低跟踪精度。为此,本文提出一种新的上下文聚合与可变形对齐框架CADTrack。首先,引入基于Mamba的特征交互模块(MFI),通过状态空间模型实现线性复杂度的高效特征交互,提升特征区分能力并降低计算成本。其次,设计上下文聚合模块(CAM),基于混合专家(MoE)的稀疏门控机制动态激活主干网络层,编码跨层互补上下文信息。最后,提出可变形对齐模块(DAM),结合可变形采样与时序传播,缓解空间错位与定位漂移问题。在五个RGBT跟踪基准上的大量实验验证了所提方法的有效性。代码已开源。

原文摘要 · Abstract (English)

RGB-Thermal (RGBT) tracking aims to exploit visible and thermal infrared modalities for robust all-weather object tracking. However, existing RGBT trackers struggle to resolve modality discrepancies, which poses great challenges for robust feature representation. This limitation hinders effective cross-modal information propagation and fusion, which significantly reduces the tracking accuracy. To address this limitation, we propose a novel Contextual Aggregation with Deformable Alignment framework called CADTrack for RGBT Tracking. To be specific, we first deploy the Mamba-based Feature Interaction (MFI) that establishes efficient feature interaction via state space models. This interaction module can operate with linear complexity, reducing computational cost and improving feature discrimination. Then, we propose the Contextual Aggregation Module (CAM) that dynamically activates backbone layers through sparse gating based on the Mixture-of-Experts (MoE). This module can encode complementary contextual information from cross-layer features. Finally, we propose the Deformable Alignment Module (DAM) to integrate deformable sampling and temporal propagation, mitigating spatial misalignment and localization drift. With the above components, our CADTrack achieves robust and accurate tracking in complex scenarios. Extensive experiments on five RGBT tracking benchmarks verify the effectiveness of our proposed method. The source code is released at https://github.com/IdolLab/CADTrack.

RGBT跟踪多模态融合可变形对齐Mamba

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。