提出统一自适应选择框架,让多模态追踪自动选最优信息。
UASTrack: A Unified Adaptive Selection Framework with Modality-Customization in Single Object Tracking
- 设计自选模块,自动识别并利用不同传感器数据特征。
- 在5个基准上表现媲美顶尖模型,仅增加187万参数和19.5亿计算量。
- 适合需要跨模态融合的实时追踪场景,如红外、深度、事件相机应用。
多模态追踪在单目标追踪中至关重要,因不同传感器可互补应对外观变化挑战。现有统一的RGB-X追踪器(X代表深度、事件或热成像)或依赖特定任务训练,或未能解决真实场景中模态自适应感知的关键问题。本文提出UASTrack,一个统一的自适应选择框架,实现模型与参数统一,并支持跨多模态追踪任务的自适应模态区分。为实现联合RGB-X图像中的模态自适应感知,设计了判别式自动选择器(DAS),可识别模态标签,区分辅助模态的数据分布。同时提出任务定制化优化适配器(TCOA),针对潜空间中不同模态特性进行优化,有效过滤冗余噪声并减轻背景干扰。在包含LasHeR、GTOT、RGBT234、VisEvent和DepthTrack在内的五个基准上进行广泛对比,覆盖RGB-T、RGB-E和RGB-D追踪场景,结果表明本方法仅引入187万额外参数和19.5亿浮点运算,即可达到与领先模型相当的性能。代码将发布于https://github.com/wanghe/UASTrack。
原文摘要 · Abstract (English)
Multi-modal tracking is essential in single-object tracking (SOT), as different sensor types contribute unique capabilities to overcome challenges caused by variations in object appearance. However, existing unified RGB-X trackers (X represents depth, event, or thermal modality) either rely on the task-specific training strategy for individual RGB-X image pairs or fail to address the critical importance of modality-adaptive perception in real-world applications. In this work, we propose UASTrack, a unified adaptive selection framework that facilitates both model and parameter unification, as well as adaptive modality discrimination across various multi-modal tracking tasks. To achieve modality-adaptive perception in joint RGB-X pairs, we design a Discriminative Auto-Selector (DAS) capable of identifying modality labels, thereby distinguishing the data distributions of auxiliary modalities. Furthermore, we propose a Task-Customized Optimization Adapter (TCOA) tailored to various modalities in the latent space. This strategy effectively filters noise redundancy and mitigates background interference based on the specific characteristics of each modality. Extensive comparisons conducted on five benchmarks including LasHeR, GTOT, RGBT234, VisEvent, and DepthTrack, covering RGB-T, RGB-E, and RGB-D tracking scenarios, demonstrate our innovative approach achieves comparative performance by introducing only additional training parameters of 1.87M and flops of 1.95G. The code will be available at https://github.com/wanghe/UASTrack.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。