针对多模态追踪难题,动态解耦融合网络提升鲁棒性。
Dynamic Disentangled Fusion Network for RGBT Tracking
- 按挑战属性解耦设计六种动态融合模型
- 在RGBT Tracking Benchmark上优于现有方法
- 适合处理光照变化、遮挡等复杂场景
RGBT追踪常面临低分辨率、外观相似、极端光照、热交叉和遮挡等挑战。现有方法多采用复杂融合模型,难以适应多样化挑战,限制性能提升。为此,我们提出动态解耦融合网络DDFNet,将融合过程解耦为六种基于挑战属性的动态融合模型,分别应对六类典型挑战。每个模型专注特定问题,提升融合能力且无需大规模训练数据。针对不同挑战的难度差异,采用动态优化策略组合多个融合单元。设计自适应聚合融合模块,结合三阶段训练算法,智能激活所需模型。此外,引入增强融合模块强化聚合特征与模态特异性特征。在基准数据集上的实验表明,DDFNet显著优于当前最优方法。
原文摘要 · Abstract (English)
RGBT tracking usually suffers from various challenging factors of low resolution, similar appearance, extreme illumination, thermal crossover and occlusion, to name a few. Existing works often study complex fusion models to handle challenging scenarios, but can not well adapt to various challenges, which might limit tracking performance. To handle this problem, we propose a novel Dynamic Disentangled Fusion Network called DDFNet, which disentangles the fusion process into several dynamic fusion models via the challenge attributes to adapt to various challenging scenarios, for robust RGBT tracking. In particular, we design six attribute-based fusion models to integrate RGB and thermal features under the six challenging scenarios respectively.Since each fusion model is to deal with the corresponding challenges, such disentangled fusion scheme could increase the fusion capacity without the dependence on large-scale training data. Considering that every challenging scenario also has different levels of difficulty, we propose to optimize the combination of multiple fusion units to form each attribute-based fusion model in a dynamic manner, which could well adapt to the difficulty of the corresponding challenging scenario. To address the issue that which fusion models should be activated in the tracking process, we design an adaptive aggregation fusion module to integrate all features from attribute-based fusion models in an adaptive manner with a three-stage training algorithm. In addition, we design an enhancement fusion module to further strengthen the aggregated feature and modality-specific features. Experimental results on benchmark datasets demonstrate the effectiveness of our DDFNet against other state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。