融合近红外与事件相机,实现暗光下的图像去模糊和噪声抑制。
Near-infrared Image Deblurring and Event Denoising with Synergistic Neuromorphic Imaging
- 利用近红外图像与事件数据的谱一致性及高阶交互进行跨模态融合。
- 在真实与模拟序列上均优于现有方法,提升暗光成像精度与鲁棒性。
- 适合自动驾驶、夜视监控等低光环境下的高保真成像应用。
近年来,夜间动态或极暗环境下的成像领域取得了显著进展,部分得益于新型传感技术的发展,如高灵敏度近红外(NIR)相机和几乎无运动模糊的事件相机。然而,不当的近红外曝光比会导致图像失真与模糊,而事件相机对夜间微弱信号敏感,易受干扰,常产生大量噪声,严重损害观测与分析效果。本文提出一种协同类神经形态成像框架,可联合实现近红外图像去模糊与事件降噪。通过利用近红外图像与可见事件之间的谱一致性和高阶交互,实现二者的同时融合、增强与自举。在真实与逼真模拟序列上的实验表明,该方法在实际场景中表现出更优的准确性和鲁棒性。本研究推动了近红外图像与事件数据的共同增强,为高保真低光成像与类神经形态推理开辟了新路径。
原文摘要 · Abstract (English)
The fields of imaging in the nighttime dynamic and other extremely dark conditions have seen impressive and transformative advancements in recent years, partly driven by the rise of novel sensing approaches, e.g., near-infrared (NIR) cameras with high sensitivity and event cameras with minimal blur. However, inappropriate exposure ratios of near-infrared cameras make them susceptible to distortion and blur. Event cameras are also highly sensitive to weak signals at night yet prone to interference, often generating substantial noise and significantly degrading observations and analysis. Herein, we develop a new framework for low-light imaging combined with NIR imaging and event-based techniques, named synergistic neuromorphic imaging, which can jointly achieve NIR image deblurring and event denoising. Harnessing cross-modal features of NIR images and visible events via spectral consistency and higher-order interaction, the NIR images and events are simultaneously fused, enhanced, and bootstrapped. Experiments on real and realistically simulated sequences demonstrate the effectiveness of our method and indicate better accuracy and robustness than other methods in practical scenarios. This study gives impetus to enhance both NIR images and events, which paves the way for high-fidelity low-light imaging and neuromorphic reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。