用神经网络实现红外多相机实时动态校准,提升精度与适应性。
Neural Real-Time Recalibration for Infrared Multi-Camera Systems
- 基于可微投影模型,直接优化内外参,支持实时更新。
- 在真实扰动下仍保持高精度,比传统方法更稳定可靠。
- 适用于车载、工业等需快速响应的多相机系统。
当前缺乏无需学习或基于神经网络的红外多相机系统实时校准方法。本文提出一种神经网络驱动的动态实时校准技术,解决时间敏感应用中多相机红外系统高精度校准难题。与传统方法相比,该方法具备更强的自适应能力,可应对在线重校准需求。核心是引入可微投影模型,将3D几何与2D图像投影直接关联,实现内外参的端到端优化。通过在相机参数上施加扰动进行动态姿态合成,模拟真实运行中的挑战,增强模型鲁棒性。提出两种变体:一种用于具备2D点本地处理能力的多相机系统,利用3D标定物的直接2D投影;另一种用于基于图像的系统,采用颜色编码投影点隐式建立对应关系。大量实验表明,本方法在有无扰动情况下均优于传统校准技术,且满足实时性要求,显著推动了实时多相机系统校准的发展。代码开源于 https://github.com/theICTlab/neural-recalibration。
原文摘要 · Abstract (English)
Currently, there are no learning-free or neural techniques for real-time recalibration of infrared multi-camera systems. In this paper, we address the challenge of real-time, highly-accurate calibration of multi-camera infrared systems, a critical task for time-sensitive applications. Unlike traditional calibration techniques that lack adaptability and struggle with on-the-fly recalibrations, we propose a neural network-based method capable of dynamic real-time calibration. The proposed method integrates a differentiable projection model that directly correlates 3D geometries with their 2D image projections and facilitates the direct optimization of both intrinsic and extrinsic camera parameters. Key to our approach is the dynamic camera pose synthesis with perturbations in camera parameters, emulating realistic operational challenges to enhance model robustness. We introduce two model variants: one designed for multi-camera systems with onboard processing of 2D points, utilizing the direct 2D projections of 3D fiducials, and another for image-based systems, employing color-coded projected points for implicitly establishing correspondence. Through rigorous experimentation, we demonstrate our method is more accurate than traditional calibration techniques with or without perturbations while also being real-time, marking a significant leap in the field of real-time multi-camera system calibration. The source code can be found at https://github.com/theICTlab/neural-recalibration
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。