用多视角参考图评估扩散模型生成的新视图质量,无需真实图像即可提升3D重建效果。
PR-IQA: Partial-Reference Image Quality Assessment for Diffusion-Based Novel View Synthesis
- 基于多视角参考图生成部分质量图,通过跨视图注意力补全为完整质量图。
- 在3DGS中仅对高置信度区域施加监督,显著减少伪影。
- 无需真实图像即可达到全参考级评估精度,适合图像质量评估与3D重建研究者。
扩散模型在稀疏视图新视图合成(NVS)中具有潜力,可生成伪真值视图以辅助3D重建流程(如3D Gaussian Splatting, 3DGS)。然而,这些合成图像常存在光照与几何不一致,直接用于监督会损害重建质量。为此,我们提出部分参考图像质量评估(PR-IQA),利用不同视角的参考图像评估扩散生成视图,无需真实标签。PR-IQA首先在重叠区域计算几何一致的部分质量图,再通过跨注意力机制融合参考视图上下文,完成质量图补全,生成稠密全图质量图。该质量图被用于指导扩散增强型3DGS管道,仅对高置信度区域施加监督。实验表明,PR-IQA超越现有IQA方法,在无真值监督下实现全参考级精度,从而更有效过滤不一致性,获得更优的3D重建与新视图合成结果。
原文摘要 · Abstract (English)
Diffusion models are promising for sparse-view novel view synthesis (NVS), as they can generate pseudo-ground-truth views to aid 3D reconstruction pipelines like 3D Gaussian Splatting (3DGS). However, these synthesized images often contain photometric and geometric inconsistencies, and their direct use for supervision can impair reconstruction. To address this, we propose Partial-Reference Image Quality Assessment (PR-IQA), a framework that evaluates diffusion-generated views using reference images from different poses, eliminating the need for ground truth. PR-IQA first computes a geometrically consistent partial quality map in overlapping regions. It then performs quality completion to inpaint this partial map into a dense, full-image map. This completion is achieved via a cross-attention mechanism that incorporates reference-view context, ensuring cross-view consistency and enabling thorough quality assessment. When integrated into a diffusion-augmented 3DGS pipeline, PR-IQA restricts supervision to high-confidence regions identified by its quality maps. Experiments demonstrate that PR-IQA outperforms existing IQA methods, achieving full-reference-level accuracy without ground-truth supervision. Thus, our quality-aware 3DGS approach more effectively filters inconsistencies, producing superior 3D reconstructions and NVS results. The project page is available at https://kakaomacao.github.io/pr-iqa-project-page/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。