用事件相机捕捉高速高动态范围的光场,实现事后对焦与深度估计。
Event fields: Capturing light fields at high speed, resolution, and dynamic range
- 通过空间/时间复用光学设计,让事件相机捕获光场信息。
- 原型验证了高速高动态场景下可实现后置对焦与深度感知。
- 适合高速摄影、机器人视觉和增强现实等需要快速响应的领域。
事件相机的像素独立响应亮度变化,相比传统帧式相机具有更低延迟、更少带宽需求和更高动态范围,因此在高速应用中日益普及。现有技术多利用其时间优势捕捉高速、高动态范围视频,而本工作提出“事件光场”(Event Fields),通过创新光学设计使事件相机实现高速光场捕捉。我们建立了相应的数学框架,提出了两种实用方案:空间复用以捕获时间导数,时间复用以捕获角度导数。为此,设计了两种互补光学系统:一个采用棱镜阵列实现空间复用,另一个使用微电机实现时间复用。通过自建仿真器与真实硬件原型评估性能,验证了两种设计的优势。该方法使典型光场功能——如后置对焦与深度估计——在高速高动态范围内得以实现,为摄影、机器人与AR/VR开启新可能,并带来渲染与机器学习的新挑战。
原文摘要 · Abstract (English)
Event cameras, which feature pixels that independently respond to changes in brightness, are becoming increasingly popular in high-speed applications due to their lower latency, reduced bandwidth requirements, and enhanced dynamic range compared to traditional frame-based cameras. Numerous imaging and vision techniques have leveraged event cameras for high-speed scene understanding by capturing high-framerate, high-dynamic range videos, primarily utilizing the temporal advantages inherent to event cameras. Additionally, imaging and vision techniques have utilized the light field-a complementary dimension to temporal information-for enhanced scene understanding. In this work, we propose "Event Fields", a new approach that utilizes innovative optical designs for event cameras to capture light fields at high speed. We develop the underlying mathematical framework for Event Fields and introduce two foundational frameworks to capture them practically: spatial multiplexing to capture temporal derivatives and temporal multiplexing to capture angular derivatives. To realize these, we design two complementary optical setups one using a kaleidoscope for spatial multiplexing and another using a galvanometer for temporal multiplexing. We evaluate the performance of both designs using a custom-built simulator and real hardware prototypes, showcasing their distinct benefits. Our event fields unlock the full advantages of typical light fields-like post-capture refocusing and depth estimation-now supercharged for high-speed and high-dynamic range scenes. This novel light-sensing paradigm opens doors to new applications in photography, robotics, and AR/VR, and presents fresh challenges in rendering and machine learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。