arXiv:2503.09396cs.CVcs.AI2025-03被引 4

用自生成数据逐步训练,提升3D高斯点云的近景视图合成质量。

Close-up-GS: Enhancing Close-Up View Synthesis in 3D Gaussian Splatting with Progressive Self-Training

  • 通过自生成数据逐步扩展模型信任区域,提升近景泛化能力。
  • 在近景视图上实现更清晰细节,优于现有方法。
  • 适合需要高质量近景渲染的3D重建与虚拟现实应用。

3D高斯点云(3DGS)在给定视角集上训练后,能生成高质量的新视角图像。然而当合成视角与训练视角差异较大时,渲染质量会显著下降,原因在于模型难以泛化到分布外场景,且在分辨率剧烈变化和遮挡情况下难以插值精细细节。近景视图生成是典型挑战:生成比训练视图更靠近物体的视角。为此,我们提出一种基于渐进式自训练的近景视图生成方法。核心思想包括:利用最新提出的3D感知生成模型See3D增强渲染视图细节;设计策略逐步扩展3DGS模型的“信任区域”,并更新See3D的参考视图集;引入细调策略,使用上述方案生成的数据对3DGS模型进行谨慎更新。我们还定义了针对近景视图的评估指标,以促进该问题的研究。在特定近景场景上的实验表明,所提方法明显优于竞争方案。

原文摘要 · Abstract (English)

3D Gaussian Splatting (3DGS) has demonstrated impressive performance in synthesizing novel views after training on a given set of viewpoints. However, its rendering quality deteriorates when the synthesized view deviates significantly from the training views. This decline occurs due to (1) the model's difficulty in generalizing to out-of-distribution scenarios and (2) challenges in interpolating fine details caused by substantial resolution changes and occlusions. A notable case of this limitation is close-up view generation--producing views that are significantly closer to the object than those in the training set. To tackle this issue, we propose a novel approach for close-up view generation based by progressively training the 3DGS model with self-generated data. Our solution is based on three key ideas. First, we leverage the See3D model, a recently introduced 3D-aware generative model, to enhance the details of rendered views. Second, we propose a strategy to progressively expand the ``trust regions'' of the 3DGS model and update a set of reference views for See3D. Finally, we introduce a fine-tuning strategy to carefully update the 3DGS model with training data generated from the above schemes. We further define metrics for close-up views evaluation to facilitate better research on this problem. By conducting evaluations on specifically selected scenarios for close-up views, our proposed approach demonstrates a clear advantage over competitive solutions.

3D高斯点云近景生成自训练视图合成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。