实现精准3D高斯点云局部编辑,解决多视角分割不一致和损失函数模糊问题。
Robust 3D-Masked Part-level Editing in 3D Gaussian Splatting with Regularized Score Distillation Sampling
- 用球谐系数建模视角依赖标签,生成跨视角一致的3D掩码。
- 引入带正则化的分数蒸馏采样损失,提升编辑精度并防止误改。
- 适合需要精细局部修改3D高斯场景的研究者与开发者使用。
近年来,3D神经表示与实例级编辑模型的发展推动了高质量3D内容的高效生成。然而,在高斯点云中实现精确的局部3D编辑仍具挑战,主要源于多视图2D部件分割不一致及分数蒸馏采样(SDS)损失固有的模糊性。为此,我们提出RoMaP框架,实现精准且剧烈的部件级修改。首先,设计鲁棒的3D掩码生成模块,结合3D-几何感知标签预测(3D-GALP),利用球谐系数(SH)建模视角相关的标签变化与软标签特性,获得跨视角一致的部件分割。其次,提出正则化分数蒸馏采样(SDS)损失,融合标准SDS损失与额外正则项;其中,通过调度潜在混合与部件(SLaMP)方法引入L1锚定损失,生成高质量的部件编辑2D图像,仅在目标区域修改并保持上下文一致性。此外,高斯先验移除等正则项进一步提升灵活性,允许超出原有上下文的修改,而鲁棒3D掩码有效避免意外编辑。实验表明,RoMaP在重建与生成的高斯场景与物体上均达到当前最优的局部3D编辑效果,定性和定量结果均领先。代码已公开于 https://janeyeon.github.io/romap。
原文摘要 · Abstract (English)
Recent advances in 3D neural representations and instance-level editing models have enabled the efficient creation of high-quality 3D content. However, achieving precise local 3D edits remains challenging, especially for Gaussian Splatting, due to inconsistent multi-view 2D part segmentations and inherently ambiguous nature of Score Distillation Sampling (SDS) loss. To address these limitations, we propose RoMaP, a novel local 3D Gaussian editing framework that enables precise and drastic part-level modifications. First, we introduce a robust 3D mask generation module with our 3D-Geometry Aware Label Prediction (3D-GALP), which uses spherical harmonics (SH) coefficients to model view-dependent label variations and soft-label property, yielding accurate and consistent part segmentations across viewpoints. Second, we propose a regularized SDS loss that combines the standard SDS loss with additional regularizers. In particular, an L1 anchor loss is introduced via our Scheduled Latent Mixing and Part (SLaMP) editing method, which generates high-quality part-edited 2D images and confines modifications only to the target region while preserving contextual coherence. Additional regularizers, such as Gaussian prior removal, further improve flexibility by allowing changes beyond the existing context, and robust 3D masking prevents unintended edits. Experimental results demonstrate that our RoMaP achieves state-of-the-art local 3D editing on both reconstructed and generated Gaussian scenes and objects qualitatively and quantitatively, making it possible for more robust and flexible part-level 3D Gaussian editing. Code is available at https://janeyeon.github.io/romap.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。