arXiv:2409.00674cs.CV2024-09ECCV被引 8

用一张图实现材质估计与光照重演,突破传统摄影立体术的采集限制。

MERLiN: Single-Shot Material Estimation and Relighting for Photometric Stereo

  • 基于注意力机制的统一网络,融合逆渲染与光照重演。
  • 在真实图像上实现高质量形状、材质重建与光照重演效果。
  • 适合需简化采集流程的三维重建与材质分析场景。

摄影立体术通常依赖多光源复杂采集以准确恢复表面法向。本文提出MERLiN,一种基于注意力的双塔网络,将单图逆渲染与光照重演整合于统一框架中。我们在合成数据上训练该物理模型,包含具有空间变化BRDF的复杂形状,并能处理间接光照影响,提升材质重建与光照重演质量。通过大量定性与定量评估,验证了该框架在真实图像上的良好泛化能力,实现了高质量的形状、材质估计与光照重演。我们对合成重演图像在摄影立体基准方法上评估其物理正确性与法向估计精度,为单张图像摄影立体术开辟新路径。本工作系统解决单图逆渲染问题,适用于合成与真实数据,有效缓解摄影立体术的数据采集挑战。

原文摘要 · Abstract (English)

Photometric stereo typically demands intricate data acquisition setups involving multiple light sources to recover surface normals accurately. In this paper, we propose MERLiN, an attention-based hourglass network that integrates single image-based inverse rendering and relighting within a single unified framework. We evaluate the performance of photometric stereo methods using these relit images and demonstrate how they can circumvent the underlying challenge of complex data acquisition. Our physically-based model is trained on a large synthetic dataset containing complex shapes with spatially varying BRDF and is designed to handle indirect illumination effects to improve material reconstruction and relighting. Through extensive qualitative and quantitative evaluation, we demonstrate that the proposed framework generalizes well to real-world images, achieving high-quality shape, material estimation, and relighting. We assess these synthetically relit images over photometric stereo benchmark methods for their physical correctness and resulting normal estimation accuracy, paving the way towards single-shot photometric stereo through physically-based relighting. This work allows us to address the single image-based inverse rendering problem holistically, applying well to both synthetic and real data and taking a step towards mitigating the challenge of data acquisition in photometric stereo.

三维重建逆渲染光照重演

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。