arXiv:2412.07371cs.CVcs.GR2024-12ICCV被引 6

用动态光照材质生成图像,提升3D重建细节质量。

PRM: Photometric Stereo based Large Reconstruction Model

  • 通过变化材质与光照生成光度立体图像,提供丰富视觉线索。
  • 在复杂外观下仍能还原精细局部结构,优于现有模型。
  • 支持实时物理渲染,适合高精度3D建模与工业应用。

我们提出PRM,一种基于光度立体的大规模三维重建模型,可生成高质量网格并保留精细局部细节。与以往依赖固定简单光照输入和监督的大型重建模型不同,PRM通过改变材质和光照来渲染光度立体图像,不仅通过丰富的光度线索提升局部细节精度,还增强了对输入图像外观变化的鲁棒性。为实现更灵活的图像渲染,我们引入实时物理基础渲染(PBR)与网格光栅化方法,支持在线渲染。此外,采用显式网格作为3D表示,使PRM能够实现可微分的PBR,从而支持多光度监督,并更好地建模镜面颜色,优化高质量几何结构。大量实验表明,即使在复杂图像外观下,PRM仍能实现显著优于其他模型的高保真重建效果。

原文摘要 · Abstract (English)

We propose PRM, a novel photometric stereo based large reconstruction model to reconstruct high-quality meshes with fine-grained local details. Unlike previous large reconstruction models that prepare images under fixed and simple lighting as both input and supervision, PRM renders photometric stereo images by varying materials and lighting for the purposes, which not only improves the precise local details by providing rich photometric cues but also increases the model robustness to variations in the appearance of input images. To offer enhanced flexibility of images rendering, we incorporate a real-time physically-based rendering (PBR) method and mesh rasterization for online images rendering. Moreover, in employing an explicit mesh as our 3D representation, PRM ensures the application of differentiable PBR, which supports the utilization of multiple photometric supervisions and better models the specular color for high-quality geometry optimization. Our PRM leverages photometric stereo images to achieve high-quality reconstructions with fine-grained local details, even amidst sophisticated image appearances. Extensive experiments demonstrate that PRM significantly outperforms other models.

3D重建光度立体物理渲染网格优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。