arXiv:2606.31065cs.CV2026-06中稿 · ECCV

用扩散模型指导材质优化,实现更真实的三维资产重建与重光照。

Diffusion-Based Material Regularization for Physics-Based Inverse Rendering

论文配图:Diffusion-Based Material Regularization for Physics-Based Inverse Rendering
图 1 · 摘自论文原文
  • 以扩散模型输出为相似性核,约束材质优化区域
  • 在三个数据集上显著提升重建与重光照质量
  • 适合需要物理一致材质的图形渲染与编辑场景

从多视角图像重建物理基础的3D资产——几何、材质和光照——是计算机图形学与视觉的核心问题,也是实现真实感重光照与编辑的前提。物理基础逆向渲染虽有精确的成像模型,但严重欠约束:缺乏强先验时,光照会混入材质,且重建对新视角和光照泛化能力差。数据驱动的扩散模型能生成视觉上合理的材质,但其预测通常不满足渲染方程,无法直接用于物理渲染。本文不取代任一范式,而是将先进扩散模型的预测视为优化中的相似性核:提出正则化损失,在扩散预测趋于恒定的表面区域惩罚材质优化偏差,同时允许优化自由匹配输入图像。基于该正则器,我们的端到端流程联合重建几何、材质与光照,产出可直接接入标准渲染管线、重光照准确的高质量资产。在Synthetic4Relight、Stanford-ORB和DTC-Synthetic数据集上,本方法在重建精度与重光照质量上均显著优于现有基线。

原文摘要 · Abstract (English)

Reconstructing physics-based 3D assets -- geometry, materials, and illumination -- from multi-view images is a core problem in computer graphics and vision, and a prerequisite for realistic relighting and editing. Physics-based inverse rendering offers an accurate image-formation model, but is severely underconstrained: without strong priors, illumination is baked into materials, and reconstructions generalize poorly to novel views and lighting. Data-driven diffusion models, in contrast, predict visually plausible materials, yet their predictions rarely satisfy the rendering equation and are not directly usable for physics-based rendering. We bridge these two paradigms rather than replacing either. Our key idea is to treat the predictions of a state-of-the-art diffusion model not as target material values but as a similarity kernel for optimization: we introduce a regularization loss that penalizes deviations in the optimized material over surface regions where the diffusion predictions are near-constant, while leaving the optimization free to match the input images. Built on this regularizer, our end-to-end pipeline jointly reconstructs geometry, materials, and illumination, yielding high-quality assets that drop into standard rendering pipelines and relight faithfully. On the Synthetic4Relight, Stanford-ORB, and DTC-Synthetic datasets, our method significantly outperforms state-of-the-art baselines in both reconstruction accuracy and relighting quality.

逆向渲染扩散模型材质重建重光照

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。