解决3D高斯点云在跨场景训练中的干扰物问题，提升重建稳定性与泛化能力。
Distractor-free Generalizable 3D Gaussian Splatting
- 基于无场景依赖的参考掩码预测与优化模块，消除干扰数据影响。
- 通过两阶段推理与干扰物修剪机制，有效去除推理时的伪影和空洞。
- 适用于未知场景下的实时推理，适合需要鲁棒泛化的3D重建应用。
我们提出DGGS，一种全新的框架，解决此前未被探索的挑战：无干扰物的可泛化3D高斯点云（3DGS）。该方法在跨场景泛化训练设置中缓解了由干扰物数据引起的3D不一致性与训练不稳定问题，同时支持在未见场景中通过参考图像实现3DGS与干扰物掩码的前向推理。DGGS在训练阶段引入无场景依赖的参考基掩码预测与精炼模块，有效消除干扰物对训练的负面影响。此外，我们设计了一种新颖的两阶段推理框架，用于参考图像评分与重选，并结合干扰物剪枝机制，进一步移除残留的干扰物3DGS原型影响。在真实数据与自建合成数据上的大量前向实验表明，DGGS在处理新干扰场景时具备出色的重建能力；此外，其泛化掩码预测精度甚至优于现有场景特定训练方法。
原文摘要 · Abstract (English)
We present DGGS, a novel framework that addresses the previously unexplored challenge: $\textbf{Distractor-free Generalizable 3D Gaussian Splatting}$ (3DGS). It mitigates 3D inconsistency and training instability caused by distractor data in the cross-scenes generalizable train setting while enabling feedforward inference for 3DGS and distractor masks from references in the unseen scenes. To achieve these objectives, DGGS proposes a scene-agnostic reference-based mask prediction and refinement module during the training phase, effectively eliminating the impact of distractor on training stability. Moreover, we combat distractor-induced artifacts and holes at inference time through a novel two-stage inference framework for references scoring and re-selection, complemented by a distractor pruning mechanism that further removes residual distractor 3DGS-primitive influences. Extensive feedforward experiments on the real and our synthetic data show DGGS's reconstruction capability when dealing with novel distractor scenes. Moreover, our generalizable mask prediction even achieves an accuracy superior to existing scene-specific training methods. Homepage is https://github.com/bbbbby-99/DGGS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。