通过视角感知修复提升多视角3D高斯图像一致性
Perspective-aware 3D Gaussian Inpainting with Multi-view Consistency
- 基于视角图自适应采样多视角,迭代优化高斯表示
- 在SPIn-NeRF和NeRFiller上分别达到26.03和29.51 dB的PSNR
- 适合需要高质量3D场景修复的虚拟现实与多媒体应用
3D高斯修复是虚拟现实与多媒体领域的重要技术,尽管预训练扩散模型已取得进展,但保持多视角一致性仍是关键挑战。本文提出PAInpainter,通过视角感知的内容传播与多视角修复结果的一致性验证,实现3D高斯表示的迭代优化。方法基于视角图自适应采样多视角,将修复图像作为先验信息传播,并在相邻视图间验证一致性,显著提升全局一致性和纹理保真度。大量实验表明,该方法优于现有技术,在SPIn-NeRF和NeRFiller数据集上的PSNR分别达26.03 dB和29.51 dB,验证了其有效性与泛化能力。
原文摘要 · Abstract (English)
3D Gaussian inpainting, a critical technique for numerous applications in virtual reality and multimedia, has made significant progress with pretrained diffusion models. However, ensuring multi-view consistency, an essential requirement for high-quality inpainting, remains a key challenge. In this work, we present PAInpainter, a novel approach designed to advance 3D Gaussian inpainting by leveraging perspective-aware content propagation and consistency verification across multi-view inpainted images. Our method iteratively refines inpainting and optimizes the 3D Gaussian representation with multiple views adaptively sampled from a perspective graph. By propagating inpainted images as prior information and verifying consistency across neighboring views, PAInpainter substantially enhances global consistency and texture fidelity in restored 3D scenes. Extensive experiments demonstrate the superiority of PAInpainter over existing methods. Our approach achieves superior 3D inpainting quality, with PSNR scores of 26.03 dB and 29.51 dB on the SPIn-NeRF and NeRFiller datasets, respectively, highlighting its effectiveness and generalization capability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。