用红外与可见光协同追踪多人,黑暗中也能准确定位。
Dynamic-Dark SLAM: RGB-Thermal Cooperative Robot Vision Strategy for Multi-Person Tracking in Both Well-Lit and Low-Light Scenes
- 用伪标注训练双模态跟踪器,实现光照不变的多人追踪。
- 亮度分类器指导切换追踪器,比融合更有效,暗光下准确率超90%。
- 提出动态黑暗建图新范式,适合夜间机器人导航与跨模态系统。
在机器人视觉中,热成像相机可在完全黑暗中识别人类,但因数据稀缺且难以区分个体,其在多人追踪(MPT)中的应用受限。本文提出一种基于共置RGB与热成像相机的协作式多人群体追踪系统,采用伪标注(边界框和身份标签)联合训练双模态追踪器。实验表明,热成像追踪器在明暗环境下均表现稳健。结果还显示,由二值亮度分类器引导的追踪器切换策略,比追踪器融合更有效。作为应用示例,提出“人即地标”图像变化模式识别方法,融合了人在暗光下的热可识别性以及静态物体(遮挡物)的外观、几何与语义特征。传统SLAM聚焦明亮环境中的静态地标建图,本研究首次提出面向完全黑暗中动态地标映射的“动态黑暗建图”新范式。此外,证明热成像与深度模态间知识迁移,可仅凭低分辨率3D LiDAR实现无RGB输入的可靠人体追踪,为跨机器人SLAM系统提供重要进展。
原文摘要 · Abstract (English)
In robot vision, thermal cameras hold great potential for recognizing humans even in complete darkness. However, their application to multi-person tracking (MPT) has been limited due to data scarcity and the inherent difficulty of distinguishing individuals. In this study, we propose a cooperative MPT system that utilizes co-located RGB and thermal cameras, where pseudo-annotations (bounding boxes and person IDs) are used to train both RGB and thermal trackers. Evaluation experiments demonstrate that the thermal tracker performs robustly in both bright and dark environments. Moreover, the results suggest that a tracker-switching strategy -- guided by a binary brightness classifier -- is more effective for information integration than a tracker-fusion approach. As an application example, we present an image change pattern recognition (ICPR) method, the ``human-as-landmark,'' which combines two key properties: the thermal recognizability of humans in dark environments and the rich landmark characteristics -- appearance, geometry, and semantics -- of static objects (occluders). Whereas conventional SLAM focuses on mapping static landmarks in well-lit environments, the present study takes a first step toward a new Human-Only SLAM paradigm, ``Dynamic-Dark SLAM,'' which aims to map even dynamic landmarks in complete darkness. Additionally, this study demonstrates that knowledge transfer between thermal and depth modalities enables reliable person tracking using low-resolution 3D LiDAR data without RGB input, contributing an important advance toward cross-robot SLAM systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。