提出TPF网络,通过任务驱动融合提升红外可见光跟踪精度。
Breaking Shallow Limits: Task-Driven Pixel Fusion for Gap-free RGBT Tracking
- 设计轻量级像素级融合适配器,结合Mamba实现低延迟实时追踪。
- 采用渐进式学习框架,先继承先进图像融合知识,再分离学习任务相关特征。
- 动态更新模板,基于最近邻选择最匹配帧,缓解外观变化影响。
现有RGBT跟踪方法常忽略融合位置对缓解模态差距的影响,而该差距是有效跟踪的关键因素。分析表明,浅层融合可缩小分布差距,但浅层网络判别力有限,难以区分任务相关信息与噪声,限制了像素级融合潜力。为此,我们提出新型任务驱动像素级融合网络TPF,通过渐进学习框架释放像素级融合在RGBT跟踪中的能力。具体地,设计轻量级像素级融合适配器(PFA),利用Mamba的线性复杂度实现实时、低延迟追踪;为增强融合能力,采用自适应多专家蒸馏先继承先进图像融合模型的知识,建立鲁棒初始化,再通过解耦表示学习实现任务相关特征融合。此外,针对初始模板与搜索帧间外观变化问题,提出最近邻动态模板更新机制,选择与当前搜索帧最接近的可靠帧作为动态模板。大量实验表明,TPF在四个公开的RGBT跟踪数据集上显著优于现有主流跟踪器。代码将在录用后发布。
原文摘要 · Abstract (English)
Current RGBT tracking methods often overlook the impact of fusion location on mitigating modality gap, which is key factor to effective tracking. Our analysis reveals that shallower fusion yields smaller distribution gap. However, the limited discriminative power of shallow networks hard to distinguish task-relevant information from noise, limiting the potential of pixel-level fusion. To break shallow limits, we propose a novel \textbf{T}ask-driven \textbf{P}ixel-level \textbf{F}usion network, named \textbf{TPF}, which unveils the power of pixel-level fusion in RGBT tracking through a progressive learning framework. In particular, we design a lightweight Pixel-level Fusion Adapter (PFA) that exploits Mamba's linear complexity to ensure real-time, low-latency RGBT tracking. To enhance the fusion capabilities of the PFA, our task-driven progressive learning framework first utilizes adaptive multi-expert distillation to inherits fusion knowledge from state-of-the-art image fusion models, establishing robust initialization, and then employs a decoupled representation learning scheme to achieve task-relevant information fusion. Moreover, to overcome appearance variations between the initial template and search frames, we presents a nearest-neighbor dynamic template updating scheme, which selects the most reliable frame closest to the current search frame as the dynamic template. Extensive experiments demonstrate that TPF significantly outperforms existing most of advanced trackers on four public RGBT tracking datasets. The code will be released upon acceptance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。