从多视角图像中提取可变神经材质，提升真实感渲染效果。
Extracting Neural Materials from Multi-view Images
- 用大型材质重建模型预测初始材质与不确定性引导
- 结合路径追踪优化，实现复杂高光材质的高质量还原
- 适合需要精确材质重建的影视与游戏渲染场景
神经材质能以紧凑通用的基底表示复杂的镜面反射和散射效应，但其获取与创作仍具挑战。本文提出NeuMatEx，一种用于从图像中提取空间变化神经材质的可微逆渲染方法。由于神经材质潜在空间具有非线性结构，直接使用传统逆渲染优化不可行。为此，我们训练了一个大型材质重建模型（LMRM），该模型从图像中直接预测初始基色、神经材质潜在表示及随机不确定性引导。这一材质先验提供良好初始化，并通过逆路径追踪优化进一步约束。预测的不确定性还通过锚定高置信度区域，防止光照与复杂镜面效应被错误嵌入材质。在合成与真实资产上的实验表明，NeuMatEx在视觉质量与材质分解方面均优于基于PBR的方法。
原文摘要 · Abstract (English)
Neural materials can represent complex specular reflections and scattering effects in a compact, universal basis. However, acquiring and authoring such materials remains challenging. We present NeuMatEx, a differentiable inverse rendering method for extracting spatially varying neural materials from images. The nonlinear structure of neural material latent spaces makes optimization with naive inverse rendering infeasible. To address this, we train a Large Material Reconstruction Model (LMRM) that directly predicts initialbase color, neural material latents, and aleatoric uncertainty guides from images. This material prior provides a good initialization and better constrains our subsequent optimization using inverse path tracing. The predicted uncertainty further helps by anchoring high-confidence regions more tightly to the LMRM prediction, preventing lighting and complex specular effects from being baked into materials. Experiments on synthetic and real assets show that NeuMatEx extracts complex materials with better visual quality and material decomposition than PBR-based methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。