在极暗环境下融合事件与RAW图像,提升噪声抑制与细节还原能力。
NEC-Diff: Noise-Robust Event-RAW Complementary Diffusion for Seeing Motion in Extreme Darkness
- 基于物理约束融合事件与RAW信号,实现双模态去噪。
- 动态估计信噪比,自适应融合特征,重建更清晰图像。
- 适用于低光场景下的高保真视觉重建,如安防监控。
在极端低光条件下高质量成像极具挑战性,光子稀缺导致严重噪声和纹理丢失,造成图像显著退化。事件相机具有120 dB高动态范围和对运动的高度敏感性,可为传统相机提供关键纹理线索。然而,现有方法多聚焦于从事件中恢复纹理,忽视图像噪声及事件本身的固有噪声,限制了光子匮乏条件下的像素准确重建。本文提出NEC-Diff,一种基于扩散的事件-RAW混合成像框架,从高度噪声信号中提取可靠信息以重建精细场景结构。核心思路包括:(1) 利用RAW图像的线性光响应特性与事件的亮度变化特性,建立物理驱动的双模态去噪约束;(2) 基于去噪结果动态估计两种模态的信噪比,引导自适应特征融合,将可靠线索注入扩散过程,实现高保真视觉重建。此外,构建了REAL数据集,包含47,800对像素对齐的低光下RAW图像、事件数据及高质量参考图像,光照范围为0.001–0.8 lux。大量实验表明,NEC-Diff在极端黑暗条件下表现优越。
原文摘要 · Abstract (English)
High-quality imaging of dynamic scenes in extremely low-light conditions is highly challenging. Photon scarcity induces severe noise and texture loss, causing significant image degradation. Event cameras, featuring a high dynamic range (120 dB) and high sensitivity to motion, serve as powerful complements to conventional cameras by offering crucial cues for preserving subtle textures. However, most existing approaches emphasize texture recovery from events, while paying little attention to image noise or the intrinsic noise of events themselves, which ultimately hinders accurate pixel reconstruction under photon-starved conditions. In this work, we propose NEC-Diff, a novel diffusion-based event-RAW hybrid imaging framework that extracts reliable information from heavily noisy signals to reconstruct fine scene structures. The framework is driven by two key insights: (1) combining the linear light-response property of RAW images with the brightness-change nature of events to establish a physics-driven constraint for robust dual-modal denoising; and (2) dynamically estimating the SNR of both modalities based on denoising results to guide adaptive feature fusion, thereby injecting reliable cues into the diffusion process for high-fidelity visual reconstruction. Furthermore, we construct the REAL (Raw and Event Acquired in Low-light) dataset which provides 47,800 pixel-aligned low-light RAW images, events, and high-quality references under 0.001-0.8 lux illumination. Extensive experiments demonstrate the superiority of NEC-Diff under extreme darkness. The project are available at: https://github.com/jinghan-xu/NEC-Diff.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。