融合事件相机与RGB图像，实现高速运动物体的高精度三维重建。
ERF-GS: Reconstructing Fast Motion from Disjoint Event-RGB Viewpoints
- 将事件数据融入3D高斯点云的优化与细化过程，提升动态建模能力。
- 在包含模糊帧和视角分离的Neu3D与Nvidia数据集上超越4DGS与E-D3DGS。
- 无需依赖同步RGB输入，适合复杂自然场景中的低帧率、强运动模糊视频。
基于深度学习的神经辐射场（NeRF）和3D高斯点云（3DGS）已显著提升动态场景三维重建的视觉精度与可扩展性。然而，快速运动物体的重建仍面临挑战；依赖传统帧式视频的方法在体育赛事和动物摄影等场景中表现不佳。本文提出事件-RGB融合高斯点云（ERF-GS）框架，将事件信息引入高斯点云管道的优化与细化阶段，充分利用高帧率事件传感器。与多数事件辅助重建方法不同，ERF-GS基于真实仿真环境构建，实现事件学习与RGB输入解耦。该设计使其不仅适用于合成数据，更可推广至包含复杂布局、低帧率和严重运动模糊的自然视频。实验表明，ERF-GS在多个版本的Neu3D与Nvidia数据集上均优于4DGS基线和同期方法E-D3DGS，且在离散事件与RGB视角下表现优异。代码已开源：https://github.com/andrewbxy/ERF-GS。
原文摘要 · Abstract (English)
Deep learning-driven representations such as neural radiance fields (NeRFs) and 3D Gaussian splatting (3DGS) have revolutionized the field of dynamic 3D scene reconstruction with improved visual precision and scalability. However, the reconstruction of fast-moving objects remains a challenge; existing methods based on conventional frame-based videos often struggle in scenarios such as sports events and animal videography. We propose an event-RGB fusion Gaussian splatting (ERF-GS) framework that integrates event information into both optimization and densification stages of the Gaussian splatting pipeline, taking advantage of novel event sensors with high frame-rate. Unlike many other event-assisted scene reconstruction methods, ERF-GS was developed using realistic simulation settings and realizes event-based learning detached from RGB inputs. This design enables its application beyond straightforward synthetic data into the realm of natural video with complex layout, low frame rates and severe motion blur. Our experiments show that ERF-GS outperforms both the 4DGS baseline and the concurrent E-D3DGS on different variants of the Neu3D and Nvidia datasets which include blurry RGB frames and disjoint RGB-event viewpoints. Our code is available at https://github.com/andrewbxy/ERF-GS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。