针对动态场景重建中运动模糊问题,提出按运动强度分区的3D高斯拆分方法。
MAPo : Motion-Aware Partitioning of Deformable 3D Gaussian Splatting for High-Fidelity Dynamic Scene Reconstruction
- 根据运动强度动态划分3D高斯,高动区域递归分段并独立建模
- 在复杂运动区域实现更清晰的渲染,峰值信噪比提升1.2dB
- 适合需要高保真动态重建的视觉生成与数字人应用
3D高斯拆分因其快速渲染和高质量静态场景重建而受到关注,正被广泛应用于多视角动态场景重建。常见方法通过学习变形场来建模基准3D高斯的时间变化,但这类方法常因单一统一模型难以表征多样运动模式,导致高动态区域出现模糊和细节丢失。为此,我们提出运动感知分区的可变形3D高斯拆分(MAPo)框架。其核心为基于动态评分的分区策略,将3D高斯分为高动态与低动态两类。对高动态3D高斯,采用递归时间分区,并为每个新时间片段复制变形网络,实现精细化运动建模;同时将低动态3D高斯视为静态以降低计算开销。然而,高动态区域的时间分区可能在帧边界引入视觉不连续。为此,我们引入跨帧一致性损失,不仅保证视觉连续性,还进一步提升渲染质量。大量实验表明,与基线方法相比,MAPo在保持相近计算成本的前提下,显著提升了渲染质量,尤其在复杂或快速运动区域表现更优。
原文摘要 · Abstract (English)
3D Gaussian Splatting, known for enabling high-quality static scene reconstruction with fast rendering, is increasingly being applied to multi-view dynamic scene reconstruction. A common strategy involves learning a deformation field to model the temporal changes of a canonical set of 3D Gaussians. However, these deformation-based methods often produce blurred renderings and lose fine motion details in highly dynamic regions due to the inherent limitations of a single, unified model in representing diverse motion patterns. To address these challenges, we introduce Motion-Aware Partitioning of Deformable 3D Gaussian Splatting (MAPo), a novel framework for high-fidelity dynamic scene reconstruction. Its core is a dynamic score-based partitioning strategy that distinguishes between high- and low-dynamic 3D Gaussians. For high-dynamic 3D Gaussians, we recursively partition them temporally and duplicate their deformation networks for each new temporal segment, enabling specialized modeling to capture intricate motion details. Concurrently, low-dynamic 3DGs are treated as static to reduce computational costs. However, this temporal partitioning strategy for high-dynamic 3DGs can introduce visual discontinuities across frames at the partition boundaries. To address this, we introduce a cross-frame consistency loss, which not only ensures visual continuity but also further enhances rendering quality. Extensive experiments demonstrate that MAPo achieves superior rendering quality compared to baselines while maintaining comparable computational costs, particularly in regions with complex or rapid motions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。