用事件相机数据实现高速运动下的实时3D渲染
EventSplat: 3D Gaussian Splatting from Moving Event Cameras for Real-time Rendering
- 结合事件相机与高斯点阵，利用事件到视频模型初始化
- 采用样条插值获取高质量相机位姿，提升重建精度
- 比现有方法快10倍，适合高速动态场景应用
我们提出一种基于事件相机数据的新型视图合成方法，通过高斯点阵实现。事件相机具有极高的时间分辨率和宽动态范围，使其在快速相机运动下仍能有效应对新视角合成挑战。初始化阶段利用事件到视频模型中的先验知识，同时通过样条插值获得事件相机轨迹上的高质量位姿，从而在提升重建质量的同时克服传统事件驱动神经辐射场（NeRF）方法的计算瓶颈。实验表明，本方法在视觉保真度和性能上均优于现有事件相机基NeRF方法，且渲染速度提升一个数量级。
原文摘要 · Abstract (English)
We introduce a method for using event camera data in novel view synthesis via Gaussian Splatting. Event cameras offer exceptional temporal resolution and a high dynamic range. Leveraging these capabilities allows us to effectively address the novel view synthesis challenge in the presence of fast camera motion. For initialization of the optimization process, our approach uses prior knowledge encoded in an event-to-video model. We also use spline interpolation for obtaining high quality poses along the event camera trajectory. This enhances the reconstruction quality from fast-moving cameras while overcoming the computational limitations traditionally associated with event-based Neural Radiance Field (NeRF) methods. Our experimental evaluation demonstrates that our results achieve higher visual fidelity and better performance than existing event-based NeRF approaches while being an order of magnitude faster to render.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。