用运动感知高斯模型实现高效动态场景重建，解决闪烁与存储问题。
MGStream: Motion-aware 3D Gaussian for Streamable Dynamic Scene Reconstruction
- 区分动静态高斯，仅对运动部分做变形与优化。
- 在真实数据集上实现更优画质、效率与时间一致性。
- 适合需要实时重建动态场景的系统开发者。
3D高斯点阵（3DGS）因其逼真的渲染效果和计算效率，在流式动态新视角合成（DNVS）中受到广泛关注。尽管渲染质量和优化策略持续改进，基于3DGS的流式动态场景重建仍存在闪烁伪影、存储效率低及难以建模新生物体的问题。为此，我们提出MGStream，采用运动相关3D高斯（3DGs）重建动态部分，使用普通3DGs处理静态部分。运动相关3DGs通过运动掩码与基于聚类的凸包算法实现，对动态部分施加刚性形变，并通过注意力机制优化以重建新生物体。由于形变与优化仅作用于运动相关3DGs，MGStream有效避免闪烁，提升存储效率。在真实数据集N3DV和MeetRoom上的大量实验表明，该方法在渲染质量、训练/存储效率和时序一致性方面均优于现有流式3DGS方法。代码已开源：https://github.com/pcl3dv/MGStream。
原文摘要 · Abstract (English)
3D Gaussian Splatting (3DGS) has gained significant attention in streamable dynamic novel view synthesis (DNVS) for its photorealistic rendering capability and computational efficiency. Despite much progress in improving rendering quality and optimization strategies, 3DGS-based streamable dynamic scene reconstruction still suffers from flickering artifacts and storage inefficiency, and struggles to model the emerging objects. To tackle this, we introduce MGStream which employs the motion-related 3D Gaussians (3DGs) to reconstruct the dynamic and the vanilla 3DGs for the static. The motion-related 3DGs are implemented according to the motion mask and the clustering-based convex hull algorithm. The rigid deformation is applied to the motion-related 3DGs for modeling the dynamic, and the attention-based optimization on the motion-related 3DGs enables the reconstruction of the emerging objects. As the deformation and optimization are only conducted on the motion-related 3DGs, MGStream avoids flickering artifacts and improves the storage efficiency. Extensive experiments on real-world datasets N3DV and MeetRoom demonstrate that MGStream surpasses existing streaming 3DGS-based approaches in terms of rendering quality, training/storage efficiency and temporal consistency. Our code is available at: https://github.com/pcl3dv/MGStream.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。