用事件数据生成高速高动态范围视频,解决失真与帧率低问题
EventHDR: from Event to High-Speed HDR Videos and Beyond

- 基于循环卷积网络+关键帧引导,缓解事件数据稀疏带来的误差积累
- 实现高速高动态范围视频重建,帧率高于此前方法,视觉质量更真实
- 构建首个真实世界配对数据集,支持后续事件到HDR研究
事件相机是新型类脑传感器,能异步捕捉场景动态。相比传统相机,其响应延迟更短、亮度敏感度更高。以往工作尝试从事件流重建高动态范围(HDR)视频,但普遍存在不真实伪影或帧率不足的问题。本文提出一种循环卷积神经网络,利用关键帧引导机制,有效防止稀疏事件数据导致的误差累积,实现高速HDR视频重建。此外,针对真实数据稀缺问题,我们设计新光学系统,采集了首个真实世界配对数据集——包含高速HDR视频与事件流,避免仿真策略带来的偏差。实验表明,本方法可生成高质量、高帧率的HDR视频。进一步验证了该框架在跨相机重建及下游任务(如目标检测、全景分割、光流估计、单目深度估计)中的潜力。
原文摘要 · Abstract (English)
Event cameras are innovative neuromorphic sensors that asynchronously capture the scene dynamics. Due to the event-triggering mechanism, such cameras record event streams with much shorter response latency and higher intensity sensitivity compared to conventional cameras. On the basis of these features, previous works have attempted to reconstruct high dynamic range (HDR) videos from events, but have either suffered from unrealistic artifacts or failed to provide sufficiently high frame rates. In this paper, we present a recurrent convolutional neural network that reconstruct high-speed HDR videos from event sequences, with a key frame guidance to prevent potential error accumulation caused by the sparse event data. Additionally, to address the problem of severely limited real dataset, we develop a new optical system to collect a real-world dataset with paired high-speed HDR videos and event streams, facilitating future research in this field. Our dataset provides the first real paired dataset for event-to-HDR reconstruction, avoiding potential inaccuracies from simulation strategies. Experimental results demonstrate that our method can generate high-quality, high-speed HDR videos. We further explore the potential of our work in cross-camera reconstruction and downstream computer vision tasks, including object detection, panoramic segmentation, optical flow estimation, and monocular depth estimation under HDR scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。