解决NeRF补全中遮挡难题，通过跨视角信息共享提升重建精度。
OccludeNeRF: Geometric-aware 3D Scene Inpainting with Collaborative Score Distillation in NeRF
- 利用扩散模型知识蒸馏时跨视角共享信息，缓解遮挡区域重建困难。
- 在多视图采样下保持一致性，渲染质量与重建保真度显著提升。
- 专为严重遮挡场景设计新数据集，适合3D重建与生成任务研究者。
随着神经辐射场（NeRF）成为强大的3D表征方法，其下游任务如基于2D图像的NeRF补全受到广泛关注。尽管已有方法在视图一致性和几何质量方面取得进展，但现有方法在处理遮挡问题时仍受限于2D先验信息不足，难以准确重建被遮挡区域。为此，本文提出一种新方法，在扩散模型知识蒸馏过程中实现跨视图信息共享，有效将遮挡区域的信息从可见视图传播至不可见视图。同时，为对齐多视图间的蒸馏方向，引入基于网格的去噪策略，并增加额外渲染视图以增强跨视图一致性。为评估方法在遮挡场景下的性能，我们构建了一个包含严重遮挡挑战性场景的新数据集，结合现有数据集进行测试。实验结果表明，相比基线方法，本方法在跨视图一致性、重建保真度方面表现更优，同时保持高渲染质量和几何真实性。
原文摘要 · Abstract (English)
With Neural Radiance Fields (NeRFs) arising as a powerful 3D representation, research has investigated its various downstream tasks, including inpainting NeRFs with 2D images. Despite successful efforts addressing the view consistency and geometry quality, prior methods yet suffer from occlusion in NeRF inpainting tasks, where 2D prior is severely limited in forming a faithful reconstruction of the scene to inpaint. To address this, we propose a novel approach that enables cross-view information sharing during knowledge distillation from a diffusion model, effectively propagating occluded information across limited views. Additionally, to align the distillation direction across multiple sampled views, we apply a grid-based denoising strategy and incorporate additional rendered views to enhance cross-view consistency. To assess our approach's capability of handling occlusion cases, we construct a dataset consisting of challenging scenes with severe occlusion, in addition to existing datasets. Compared with baseline methods, our method demonstrates better performance in cross-view consistency and faithfulness in reconstruction, while preserving high rendering quality and fidelity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。