研究可见光与热成像融合的增强技术,提升夜间监控目标检测精度
Augmentation techniques for video surveillance in the visible and thermal spectral range

- 提出多光谱数据增强方法,融合可见光与热成像特征
- 验证不同光照与温度变化下模型鲁棒性,提升跨模态识别能力
- 适合智能安防、自动驾驶等需全天候感知的场景
在智能视频监控中,摄像头需在昼夜条件下持续记录图像序列。通常需使用不同传感器:长波红外相机全天工作,而可见光相机仅在白天记录。本研究聚焦于基于卷积神经网络的多光谱目标检测任务。可见光图像包含丰富的颜色和纹理信息,但缺乏物体热辐射特征;而热成像则反之。尽管颜色有助于分类,但光照变化和传感器差异仍是挑战。同时,高质量热成像数据集稀缺,限制了深度网络训练。因此,利用可见光数据辅助训练具有优势,尤其当待评估数据同时包含可见光与红外信息时。然而,热辐射、形状和颜色变化对分类精度的影响尚不明确。为深入理解CNN如何决策及学习不同传感器输入,本文系统研究了多种数据增强技术的适用性与鲁棒性。
原文摘要 · Abstract (English)
In intelligent video surveillance, cameras record image sequences during day and night. Commonly, this demands different sensors. To achieve a better performance it is not unusual to combine them. We focus on the case that a long-wave infrared camera records continuously and in addition to this, another camera records in the visible spectral range during daytime and an intelligent algorithm supervises the picked up imagery. More accurate, our task is multispectral CNN-based object detection. At first glance, images originating from the visible spectral range differ between thermal infrared ones in the presence of color and distinct texture information on the one hand and in not containing information about thermal radiation that emits from objects on the other hand. Although color can provide valuable information for classification tasks, effects such as varying illumination and specialties of different sensors still represent significant problems. Anyway, obtaining sufficient and practical thermal infrared datasets for training a deep neural network poses still a challenge. That is the reason why training with the help of data from the visible spectral range could be advantageous, particularly if the data, which has to be evaluated contains both visible and infrared data. However, there is no clear evidence of how strongly variations in thermal radiation, shape, or color information influence classification accuracy. To gain deeper insight into how Convolutional Neural Networks make decisions and what they learn from different sensor input data, we investigate the suitability and robustness of different augmentation techniques...
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。