用多视角图像生成抗光照变化的逼真材质贴图,适合3D资产快速创建。
MaterialMVP: Illumination-Invariant Material Generation via Multi-view PBR Diffusion
- 通过参考注意力提取图像特征,实现可控纹理生成。
- 在不同光照和视角下保持材质一致性,生成效果更稳定。
- 分通道优化颜色与金属粗糙度贴图,对齐精度高,适合工业级应用。
基于物理的渲染(PBR)已成为现代计算机图形学的核心,支持3D场景中真实材质表现与光照交互。本文提出MaterialMVP,一种从3D网格和图像提示端到端生成PBR贴图的新模型,解决多视角材质合成中的关键挑战。该方法利用参考注意力从输入参考图像中提取并编码信息性潜在表示,实现直观且可控制的纹理生成。我们引入一致性正则化训练策略,强化不同视角与光照条件下的稳定性,确保生成结果具有光照不变性与几何一致性。此外,提出双通道材质生成机制,分别优化漫反射(albedo)与金属-粗糙度(MR)贴图,通过多通道对齐注意力保持与输入图像的精确空间对齐。可学习的材质嵌入进一步捕捉albedo与MR的独特属性。实验表明,本模型在多种光照场景下生成具有真实行为的PBR贴图,相比现有方法在一致性和质量上均有提升,适用于可扩展的3D资产创建。
原文摘要 · Abstract (English)
Physically-based rendering (PBR) has become a cornerstone in modern computer graphics, enabling realistic material representation and lighting interactions in 3D scenes. In this paper, we present MaterialMVP, a novel end-to-end model for generating PBR textures from 3D meshes and image prompts, addressing key challenges in multi-view material synthesis. Our approach leverages Reference Attention to extract and encode informative latent from the input reference images, enabling intuitive and controllable texture generation. We also introduce a Consistency-Regularized Training strategy to enforce stability across varying viewpoints and illumination conditions, ensuring illumination-invariant and geometrically consistent results. Additionally, we propose Dual-Channel Material Generation, which separately optimizes albedo and metallic-roughness (MR) textures while maintaining precise spatial alignment with the input images through Multi-Channel Aligned Attention. Learnable material embeddings are further integrated to capture the distinct properties of albedo and MR. Experimental results demonstrate that our model generates PBR textures with realistic behavior across diverse lighting scenarios, outperforming existing methods in both consistency and quality for scalable 3D asset creation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。