用神经正则化修复多视角特征不一致导致的3D语义噪声
NRGS: Neural Regularization for Robust 3D Semantic Gaussian Splatting
- 在3D高斯上直接使用条件MLP，结合几何与外观信息修正语义错误
- 实验显示该方法显著提升语义精度，且无需额外预处理或高计算开销
- 适合需要高鲁棒性3D语义重建的研究者，尤其关注效率与质量平衡
我们提出一种神经正则化方法，用于优化由多视角不一致2D特征提升产生的噪声3D语义场，以获得准确且鲁棒的3D语义高斯点云。从视觉基础模型提取的2D特征因缺乏跨视角约束而存在多视角不一致性。直接将这些不一致特征提升至3D高斯会导致语义场噪声，降低下游任务性能。现有方法或在预处理阶段追求多视角特征一致性，或通过改进优化策略缓解噪声，但常伴随预处理时间增加或高昂计算开销。相比之下，我们设计了一种方差感知的条件MLP，直接作用于3D高斯，利用其几何与外观属性纠正3D空间中的语义错误。在多个数据集上的实验表明，该方法有效提升提升语义的准确性，提供一种高效且有效的鲁棒3D语义高斯点云生成方案。
原文摘要 · Abstract (English)
We propose a neural regularization method that refines the noisy 3D semantic field produced by lifting multi-view inconsistent 2D features, in order to obtain an accurate and robust 3D semantic Gaussian Splatting. The 2D features extracted from vision foundation models suffer from multi-view inconsistency due to a lack of cross-view constraints. Lifting these inconsistent features directly into 3D Gaussians results in a noisy semantic field, which degrades the performance of downstream tasks. Previous methods either focus on obtaining consistent multi-view features in the preprocessing stage or aim to mitigate noise through improved optimization strategies, often at the cost of increased preprocessing time or expensive computational overhead. In contrast, we introduce a variance-aware conditional MLP that operates directly on the 3D Gaussians, leveraging their geometric and appearance attributes to correct semantic errors in 3D space. Experiments on different datasets show that our method enhances the accuracy of lifted semantics, providing an efficient and effective approach to robust 3D semantic Gaussian Splatting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。