用语义信息约束动态3D高斯点云变形,让物体保持形状一致。
SemDynReg: Semantics-Guided Deformation Regularization for Dynamic 3D Gaussian Splatting

- 通过SAM提取分割掩码,构建物体ID映射图。
- 对每个物体的top-k高斯点施加位置/尺度/旋转一致性正则。
- 适合需要精准物体变形的动态3D渲染任务。
可变形3D高斯溅射(3DGS)在多种3D应用中成为高效渲染动态场景的方法。然而,现有基于变形场的方法普遍缺乏显式物体级建模,常导致单个物体内部高斯点变形不一致,并引发不同物体间的异常耦合。为此,我们提出一种语义引导的框架,在物体级别施加动态正则化,以实现空间上一致的物体级变形。具体而言,首先利用分割一切模型(SAM)提取分割掩码,并从输入图像中提取语义特征;通过特征相关性匹配预定义物体字典,构建物体ID映射图。基于该映射图,识别每个物体对应的像素级top-k贡献高斯点,并对其变形参数(位置、尺度、旋转)施加一致性正则。与以往无显式物体约束的学习变形场方法不同,本方法引入语义线索指导物体级别的变形行为。实验表明,语义感知正则显著提升物体级变形一致性,在渲染质量上优于基线方法,取得更高的PSNR和SSIM,更低的LPIPS。项目页面见 https://dyn-reg-3dgs.github.io/。
原文摘要 · Abstract (English)
Deformable 3D Gaussian Splatting (3DGS) has emerged as an efficient approach for rendering dynamic scenes in a wide range of 3D applications. However, existing deformation field-based approaches largely lack explicit object-level modeling, often resulting in inconsistent Gaussian deformations within individual objects and unwanted coupling between different objects. To address this limitation, we introduce a semantics-guided framework that enforces dynamic regularization at the object level, aiming to achieve spatially consistent object-wise deformation. Specifically, we first extract segmentation masks using the Segment Anything Model (SAM) and derive semantic features from input images. An object-ID map is then constructed via feature relevance matching with a predefined object dictionary. Guided by this object-ID map, we identify the pixel-wise top-k contributing Gaussians for each object and impose consistency regularization on their deformation parameters, including position, scale, and rotation. Unlike prior methods that learn deformation fields without explicit object-level constraints, our approach incorporates semantic cues to guide deformation behavior at the object level. Experimental results demonstrate that our semantics-aware regularization improves object-level deformation consistency and outperforms baseline methods in rendering quality, achieving higher PSNR and SSIM and lower LPIPS in dynamic 3DGS rendering. Our project page is available at https://dyn-reg-3dgs.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。