用多帧节点引导提升动态场景重建的时空一致性。
Mango-GS: Enhancing Spatio-Temporal Consistency in Dynamic Scenes Reconstruction using Multi-Frame Node-Guided 4D Gaussian Splatting
- 通过时序Transformer在短帧窗口内建模运动依赖。
- 控制节点解耦位置与隐码，防止大运动下的对应漂移。
- 支持实时渲染，适合交互式动态场景重建。
重建具有照片级细节和强时序一致性的动态3D场景仍是重大挑战。现有基于高斯点阵的动态场景建模方法通常依赖逐帧优化，易过拟合于瞬时状态而非捕捉底层运动动态。为此，我们提出Mango-GS，一种多帧、节点引导的4D高斯点阵重建框架。该框架采用时序Transformer在短帧窗口内建模运动依赖，生成时序一致的形变；为提升效率，时序建模仅作用于稀疏控制节点。每个节点由解耦的规范位置与隐码表示，提供稳定的语义锚点以实现运动传播，并在大运动下防止对应漂移。框架端到端训练，辅以输入掩码策略及两项多帧损失以增强鲁棒性。大量实验表明，Mango-GS在重建质量上达到当前最优水平，同时支持实时渲染，可实现动态场景的高保真重建与交互式呈现。
原文摘要 · Abstract (English)
Reconstructing dynamic 3D scenes with photorealistic detail and strong temporal coherence remains a significant challenge. Existing Gaussian splatting approaches for dynamic scene modeling often rely on per-frame optimization, which can overfit to instantaneous states instead of capturing underlying motion dynamics. To address this, we present Mango-GS, a multi-frame, node-guided framework for high-fidelity 4D reconstruction. Mango-GS leverages a temporal Transformer to model motion dependencies within a short window of frames, producing temporally consistent deformations. For efficiency, temporal modeling is confined to a sparse set of control nodes. Each node is represented by a decoupled canonical position and a latent code, providing a stable semantic anchor for motion propagation and preventing correspondence drift under large motion. Our framework is trained end-to-end, enhanced by an input masking strategy and two multi-frame losses to improve robustness. Extensive experiments demonstrate that Mango-GS achieves state-of-the-art reconstruction quality and real-time rendering speed, enabling high-fidelity reconstruction and interactive rendering of dynamic scenes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。