arXiv:2609.08332cs.CV2026-09

用事件流驱动3D动态场景重建,实现极低帧率下的真实运动渲染。

EdMCGS: Event-Driven Markov Chain Gaussian Splatting for Extreme-Low-Frame-Rate Dynamic Scene Reconstruction

论文配图:EdMCGS: Event-Driven Markov Chain Gaussian Splatting for Extreme-Low-Frame-Rate Dynamic Scene Reconstruction
图 1 · 摘自论文原文
  • 以事件流为驱动构建马尔可夫链,直接生成中间帧运动。
  • 在真实数据上超越基线方法,仅用更少高斯点实现实时渲染。
  • 适合低帧率视频重建与实时3D场景渲染任务。

我们提出EdMCGS(事件驱动马尔可夫链高斯点渲染),一种端到端方法,从极低帧率的RGB图像和事件流中重建动态3D场景,并可在任意时间戳渲染。依赖单一RGB图像的方法因连续帧间缺乏证据而产生大量伪影。为补充缺失信息,我们将场景运动建模为事件驱动的马尔可夫链:稀疏的RGB帧锚定各自时刻的状态,而区间内的事件驱动状态转移。由于转移过程读取当前区间的事件,推理时保持活跃,直接由事件生成3D高斯点的中间运动,而非插值,这与以往仅在训练时使用事件监督的方法不同。状态由一组紧凑的控制点携带,每个点由其图像投影邻域内采样的事件驱动,同时引入时间局部等距项保证运动局部刚性。在合成与真实场景上的实验表明,EdMCGS优于基于RGB和基于事件的基线方法,且以远少于最强事件基线的高斯点数实现实时渲染。源码与新数据集已发布于https://github.com/joseclipse/EdMCGS。

原文摘要 · Abstract (English)

We present EdMCGS (Event-driven Markov chain Gaussian Splatting), an end-to-end method for reconstructing dynamic 3D scenes from extreme-low-frame-rate RGB together with an event stream, which can then be rendered at any intermediate timestamp. Methods relying solely on RGB images generate numerous artifacts due to the lack of evidence from between consecutive frames. To supply this missing evidence, we model the scene motion as an event-driven Markov chain, in which the sparse RGB frames anchor the state at their own timestamps while the events recorded within an interval drive the transition across it. Since the transition reads the events of the current interval, it remains active at inference and produces the in-between motion of the 3D Gaussians directly from the events rather than by interpolation, which sets our method apart from prior work that uses events only as training-time supervision. The state is carried by a compact set of control points, each driven by the events sampled in the neighborhood of its own image projection, and a temporal local isometry term keeps the propagated motion locally rigid. Experiments on synthetic and real-world scenes show that EdMCGS outperforms both RGB-based and event-based baselines, while rendering in real time with far fewer Gaussians than the strongest event-based baseline. We release our source code and a new dataset at https://github.com/joseclipse/EdMCGS.

3D重建事件相机动态场景

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。