用事件相机数据自适应调节全光照范围图像亮度,提升成像灵活性。
SEE: See Everything Every Time -- Adaptive Brightness Adjustment for Broad Light Range Images via Events
- 通过事件数据构建亮度词典,结合提示实现像素级亮度调节。
- 在61万张跨1000倍照度变化的图像上验证,显著优于传统方法。
- 适合需要灵活调光的机器人、自动驾驶等实时视觉场景。
事件相机具有超过120dB的高动态范围,能鲁棒地记录不同光照条件下的细节变化,包括低光与高光环境。然而,现有研究主要聚焦于低光图像增强,忽略了正常及高照度条件下图像增强与亮度调节问题。为此,我们提出新问题:如何利用事件数据对广泛光照条件下的图像进行增强与自适应亮度调整?为此,我们构建了新的数据集SEE-600K,包含610,126张图像及对应事件,覆盖202个场景,每个场景平均含4种光照条件,照度变化超过1000倍。我们提出一个框架,通过提示引导事件数据实现平滑亮度调节:利用传感器模式捕捉颜色,采用交叉注意力将事件建模为亮度词典,调整图像动态范围生成宽光范围表示(BLR),并基于亮度提示在像素级解码。实验表明,该方法不仅在低光增强数据集上表现优异,更在使用SEE-600K数据集的广域光照增强任务中展现强鲁棒性。此外,支持像素级亮度调节,为后期处理提供灵活性,推动更多成像应用发展。数据集与代码已公开于https://github.com/yunfanLu/SEE。
原文摘要 · Abstract (English)
Event cameras, with a high dynamic range exceeding $120dB$, significantly outperform traditional embedded cameras, robustly recording detailed changing information under various lighting conditions, including both low- and high-light situations. However, recent research on utilizing event data has primarily focused on low-light image enhancement, neglecting image enhancement and brightness adjustment across a broader range of lighting conditions, such as normal or high illumination. Based on this, we propose a novel research question: how to employ events to enhance and adaptively adjust the brightness of images captured under broad lighting conditions? To investigate this question, we first collected a new dataset, SEE-600K, consisting of 610,126 images and corresponding events across 202 scenarios, each featuring an average of four lighting conditions with over a 1000-fold variation in illumination. Subsequently, we propose a framework that effectively utilizes events to smoothly adjust image brightness through the use of prompts. Our framework captures color through sensor patterns, uses cross-attention to model events as a brightness dictionary, and adjusts the image's dynamic range to form a broad light-range representation (BLR), which is then decoded at the pixel level based on the brightness prompt. Experimental results demonstrate that our method not only performs well on the low-light enhancement dataset but also shows robust performance on broader light-range image enhancement using the SEE-600K dataset. Additionally, our approach enables pixel-level brightness adjustment, providing flexibility for post-processing and inspiring more imaging applications. The dataset and source code are publicly available at: https://github.com/yunfanLu/SEE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。