融合可见光与红外图像,提升小目标多对象跟踪精度。
Heterogeneous Graph Transformer for Multiple Tiny Object Tracking in RGB-T Videos
- 用异构图变压器融合多模态时空特征。
- 在新基准上实现82.3% MOTA和79.1% ID-F1。
- 适合遥感、安防中弱特征小目标跟踪场景。
由于小目标外观微弱、特征有限,多目标跟踪极具挑战。现有算法多聚焦单模态场景,忽视多传感器对小目标的互补信息。为此,我们提出基于异构图变压器的多小目标跟踪框架HGT-Track。首先通过Transformer编码器提取多模态图像特征;随后利用异构图变压器聚合跨模态的空间与时间信息,生成检测与跟踪特征;引入重检测模块(ReDet)以保持跨模态一致性,确保轨迹连续性。此外,本文构建首个基于可见-红外(RGB-T)融合的小目标多对象跟踪基准数据集VT-Tiny-MOT。在该数据集上的大量实验表明,所提方法在MOTA(82.3%)和ID-F1(79.1%)上优于现有先进方法。代码与数据集将公开于https://github.com/xuqingyu26/HGTMT。
原文摘要 · Abstract (English)
Tracking multiple tiny objects is highly challenging due to their weak appearance and limited features. Existing multi-object tracking algorithms generally focus on single-modality scenes, and overlook the complementary characteristics of tiny objects captured by multiple remote sensors. To enhance tracking performance by integrating complementary information from multiple sources, we propose a novel framework called {HGT-Track (Heterogeneous Graph Transformer based Multi-Tiny-Object Tracking)}. Specifically, we first employ a Transformer-based encoder to embed images from different modalities. Subsequently, we utilize Heterogeneous Graph Transformer to aggregate spatial and temporal information from multiple modalities to generate detection and tracking features. Additionally, we introduce a target re-detection module (ReDet) to ensure tracklet continuity by maintaining consistency across different modalities. Furthermore, this paper introduces the first benchmark VT-Tiny-MOT (Visible-Thermal Tiny Multi-Object Tracking) for RGB-T fused multiple tiny object tracking. Extensive experiments are conducted on VT-Tiny-MOT, and the results have demonstrated the effectiveness of our method. Compared to other state-of-the-art methods, our method achieves better performance in terms of MOTA (Multiple-Object Tracking Accuracy) and ID-F1 score. The code and dataset will be made available at https://github.com/xuqingyu26/HGTMT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。