arXiv:2507.04749cs.CV2025-07

从多视角图像中恢复高精度3D形状与材质,支持可编辑可重光照。

MatDecompSDF: High-Fidelity 3D Shape and PBR Material Decomposition from Multi-View Images

  • 用神经SDF+材质场+光照MLP联合优化,实现几何、材质、光照解耦。
  • 在DTU等数据集上重建精度超越现有方法,材质还原更真实。
  • 输出可编辑的PBR资产,适合游戏与影视数字内容制作。

我们提出MatDecompSDF,一种从多视角图像中恢复高保真3D形状并分解其物理基础材质属性的新框架。逆渲染的核心挑战在于从二维观测中解耦几何、材质和光照。本方法通过联合优化三个神经组件:用于表示复杂几何的神经符号距离函数(SDF)、用于预测PBR材质参数(反照率、粗糙度、金属度)的空间变化神经场,以及基于MLP的未知环境光照模型。关键在于引入基于物理的可微分渲染层,将三维属性与输入图像关联,实现端到端优化。我们设计了多项物理先验与几何正则化,包括材质平滑损失和Eikonal损失,有效约束问题并实现鲁棒分解。在合成数据及真实世界数据集(如DTU)上的大量实验表明,MatDecompSDF在几何精度、材质保真度和新视角合成方面均优于当前最优方法。更重要的是,该方法生成的可编辑、可重光照资产能无缝集成至标准图形管线,验证其在数字内容创作中的实际价值。

原文摘要 · Abstract (English)

We present MatDecompSDF, a novel framework for recovering high-fidelity 3D shapes and decomposing their physically-based material properties from multi-view images. The core challenge of inverse rendering lies in the ill-posed disentanglement of geometry, materials, and illumination from 2D observations. Our method addresses this by jointly optimizing three neural components: a neural Signed Distance Function (SDF) to represent complex geometry, a spatially-varying neural field for predicting PBR material parameters (albedo, roughness, metallic), and an MLP-based model for capturing unknown environmental lighting. The key to our approach is a physically-based differentiable rendering layer that connects these 3D properties to the input images, allowing for end-to-end optimization. We introduce a set of carefully designed physical priors and geometric regularizations, including a material smoothness loss and an Eikonal loss, to effectively constrain the problem and achieve robust decomposition. Extensive experiments on both synthetic and real-world datasets (e.g., DTU) demonstrate that MatDecompSDF surpasses state-of-the-art methods in geometric accuracy, material fidelity, and novel view synthesis. Crucially, our method produces editable and relightable assets that can be seamlessly integrated into standard graphics pipelines, validating its practical utility for digital content creation.

3D重建材质分解PBR逆渲染

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。