arXiv:2410.16995cs.CVcs.RO2024-10中稿 · Applied Optics被引 5

用事件相机同时捕捉运动与曝光信号,提升复杂光照下的3D重建质量。

E-3DGS: Gaussian Splatting with Exposure and Motion Events

  • 通过硬件调节透射率,让事件相机同步采集运动和曝光事件
  • 在低光或过曝条件下,重建细节优于传统帧相机
  • 支持高质量、快速及混合三种模式,适合动态场景重建

在理想成像条件下实现3D重建已得到广泛研究,但在真实场景中,运动模糊与光照不足常限制帧基相机的成像质量。为此,我们在硬件层面引入透射率调节装置,使事件相机能够同时捕获运动事件(由相机或物体移动触发)与曝光事件(通过控制相机曝光生成)。在高速运动时,系统记录运动事件;在慢速运动时,利用曝光事件重建灰度图像,用于高质量训练和优化基于事件的3D高斯点云(3DGS)。本框架支持三种模式:使用曝光事件的高质量重建、依赖运动事件的快速重建,以及先用曝光事件初始化再结合高速运动事件的平衡混合重建。在EventNeRF数据集上,曝光事件显著提升细节重建效果,优于仅依赖运动事件的方法,并在低光和过曝条件下超越帧基相机。我们还构建了E-3D,一个包含曝光事件、运动事件、相机标定参数和稀疏点云的真实世界3D数据集。所提方法在事件基3D重建中实现更快更优的效果,且成本低于融合事件与RGB数据的方法。E-3DGS在挑战性条件下表现出更强鲁棒性,对硬件要求更低,树立了事件基3D重建新基准。代码与数据集将开源于https://github.com/MasterHow/E-3DGS。

原文摘要 · Abstract (English)

Achieving 3D reconstruction from images captured under optimal conditions has been extensively studied in the vision and imaging fields. However, in real-world scenarios, challenges such as motion blur and insufficient illumination often limit the performance of standard frame-based cameras in delivering high-quality images. To address these limitations, we incorporate a transmittance adjustment device at the hardware level, enabling event cameras to capture both motion and exposure events for diverse 3D reconstruction scenarios. Motion events (triggered by camera or object movement) are collected in fast-motion scenarios when the device is inactive, while exposure events (generated through controlled camera exposure) are captured during slower motion to reconstruct grayscale images for high-quality training and optimization of event-based 3D Gaussian Splatting (3DGS). Our framework supports three modes: High-Quality Reconstruction using exposure events, Fast Reconstruction relying on motion events, and Balanced Hybrid optimizing with initial exposure events followed by high-speed motion events. On the EventNeRF dataset, we demonstrate that exposure events significantly improve fine detail reconstruction compared to motion events and outperform frame-based cameras under challenging conditions such as low illumination and overexposure. Furthermore, we introduce EME-3D, a real-world 3D dataset with exposure events, motion events, camera calibration parameters, and sparse point clouds. Our method achieves faster and higher-quality reconstruction than event-based NeRF and is more cost-effective than methods combining event and RGB data. E-3DGS sets a new benchmark for event-based 3D reconstruction with robust performance in challenging conditions and lower hardware demands. The source code and dataset will be available at https://github.com/MasterHow/E-3DGS.

3D重建事件相机高斯溅射低光成像

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。