用热成像与激光雷达融合追踪行人,光照变化也不怕。
A Dual-Stream Transformer Architecture for Illumination-Invariant TIR-LiDAR Person Tracking
- 双流Transformer架构融合热成像与深度数据
- 在复杂光照下达到AO 0.700、SR 58.7%的性能
- 适合全天候机器人自主跟人场景
鲁棒的人体追踪对在多变环境中运行的自主移动机器人至关重要。尽管RGB-D追踪精度高,但在极端光照条件下(如完全黑暗或强逆光)性能严重下降。为实现全天候鲁棒性,本文提出一种基于热成像与深度(TIR-D)传感器的新型追踪架构,利用SLAM兼容机器人标准配置的激光雷达和热成像相机。TIR-D追踪的一大挑战是缺乏标注的多模态数据集。为此,我们引入一种序列知识迁移策略,将大规模热成像预训练模型的结构先验迁移到TIR-D领域。通过采用微调差异学习率策略——即“细粒度差异学习率策略”——有效保留预训练特征提取能力,同时快速适应几何深度信息。实验表明,所提出的TIR-D追踪器性能优越,平均重叠(AO)达0.700,成功率(SR)为58.7%,显著优于传统RGB迁移及单模态基线方法。该方法为全天候机器人人体跟随应用提供了一种高效且资源节约的解决方案。
原文摘要 · Abstract (English)
Robust person tracking is a critical capability for autonomous mobile robots operating in diverse and unpredictable environments. While RGB-D tracking has shown high precision, its performance severely degrades under challenging illumination conditions, such as total darkness or intense backlighting. To achieve all-weather robustness, this paper proposes a novel Thermal-Infrared and Depth (TIR-D) tracking architecture that leverages the standard sensor suite of SLAM-capable robots, namely LiDAR and TIR cameras. A major challenge in TIR-D tracking is the scarcity of annotated multi-modal datasets. To address this, we introduce a sequential knowledge transfer strategy that evolves structural priors from a large-scale thermal-trained model into the TIR-D domain. By employing a differential learning rate strategy -- referred to as ``Fine-grained Differential Learning Rate Strategy'' -- we effectively preserve pre-trained feature extraction capabilities while enabling rapid adaptation to geometric depth cues. Experimental results demonstrate that our proposed TIR-D tracker achieves superior performance, with an Average Overlap (AO) of 0.700 and a Success Rate (SR) of 58.7\%, significantly outperforming conventional RGB-transfer and single-modality baselines. Our approach provides a practical and resource-efficient solution for robust human-following in all-weather robotics applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。