从单图估计真实感材质，光照复杂也能准。
DualMat: PBR Material Estimation via Coherent Dual-Path Diffusion
- 双路径扩散模型，分别优化颜色和材质参数。
- 颜色估计精度提升28%，金属粗糙度误差降39%。
- 适合图像转3D、高分辨率及多视角应用。
我们提出DualMat，一种新颖的双路径扩散框架，用于在复杂光照条件下从单张图像估计基于物理的渲染（PBR）材质。该方法在两个不同的潜在空间中运行：一个利用预训练视觉知识的RGB潜在空间优化反照率，另一个在专为精确金属度和粗糙度估计设计的紧凑潜在空间中工作。为确保两路径间预测的一致性，训练过程中引入特征蒸馏。采用修正流（rectified flow）提升效率，减少推理步数同时保持高质量。通过基于块的估计与跨视角注意力机制，框架可扩展至高分辨率和多视角输入，实现与图像到3D流程的无缝集成。DualMat在Objaverse和真实世界数据上均达到领先性能，反照率估计精度提升最高达28%，金属-粗糙度预测误差降低39%。
原文摘要 · Abstract (English)
We present DualMat, a novel dual-path diffusion framework for estimating Physically Based Rendering (PBR) materials from single images under complex lighting conditions. Our approach operates in two distinct latent spaces: an albedo-optimized path leveraging pretrained visual knowledge through RGB latent space, and a material-specialized path operating in a compact latent space designed for precise metallic and roughness estimation. To ensure coherent predictions between the albedo-optimized and material-specialized paths, we introduce feature distillation during training. We employ rectified flow to enhance efficiency by reducing inference steps while maintaining quality. Our framework extends to high-resolution and multi-view inputs through patch-based estimation and cross-view attention, enabling seamless integration into image-to-3D pipelines. DualMat achieves state-of-the-art performance on both Objaverse and real-world data, significantly outperforming existing methods with up to 28% improvement in albedo estimation and 39% reduction in metallic-roughness prediction errors.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。