用视频生成模型修复3D高斯点云的视图缺陷,提升重建质量。
ArtifactWorld: Scaling 3D Gaussian Splatting Artifact Restoration via Video Generation Models

- 构建107.5万对视频数据集,分类细化3DGS退化现象
- 通过热力图定位缺陷并引导修复,实现时空一致重建
- 适合需要高质量3D重建的工业与科研场景
3D高斯点云(3DGS)虽能实现高保真实时渲染,但在稀疏视角下易出现几何与光照失真。现有生成式修复方法常因时间一致性不足、缺乏显式空间约束及训练数据匮乏,导致多视角不一致、几何幻觉错误,且泛化能力有限。本文提出ArtifactWorld框架,通过系统性数据扩展与统一双模型架构解决上述问题。为突破数据瓶颈,建立细粒度的3DGS退化现象分类体系,构建包含107.5万对多样配对视频片段的训练集,增强模型鲁棒性。架构上,采用视频扩散模型主干,利用同构预测器生成退化热力图以定位结构缺陷,并通过抗伪影三重融合机制,实现基于强度引导的原生自注意力时空修复。大量实验证明,ArtifactWorld在稀疏新视角合成与鲁棒3D重建任务中达到当前最优性能。代码与数据集将公开。
原文摘要 · Abstract (English)
3D Gaussian Splatting (3DGS) delivers high-fidelity real-time rendering but suffers from geometric and photometric degradations under sparse-view constraints. Current generative restoration approaches are often limited by insufficient temporal coherence, a lack of explicit spatial constraints, and a lack of large-scale training data, resulting in multi-view inconsistencies, erroneous geometric hallucinations, and limited generalization to diverse real-world artifact distributions. In this paper, we present ArtifactWorld, a framework that resolves 3DGS artifact repair through systematic data expansion and a homogeneous dual-model paradigm. To address the data bottleneck, we establish a fine-grained phenomenological taxonomy of 3DGS artifacts and construct a comprehensive training set of 107.5K diverse paired video clips to enhance model robustness. Architecturally, we unify the restoration process within a video diffusion backbone, utilizing an isomorphic predictor to localize structural defects via an artifact heatmap. This heatmap then guides the restoration through an Artifact-Aware Triplet Fusion mechanism, enabling precise, intensity-guided spatio-temporal repair within native self-attention. Extensive experiments demonstrate that ArtifactWorld achieves state-of-the-art performance in sparse novel view synthesis and robust 3D reconstruction. Code and dataset will be made public.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。