用文本生成可物理渲染的程序化材质，支持光照重演
TexPro: Text-guided PBR Texturing with Procedural Material Modeling
- 通过文本生成多视角图像，再用可微分程序化材质优化纹理
- 在ShapeNet数据集上实现比当前最佳方法高12.3%的PSNR
- 适合需要真实光照效果的3D内容创作人员
本文提出TexPro，一种基于文本提示为输入3D网格生成高保真材质的新方法。与以往仅生成带烘焙光照的RGB纹理的方法不同，TexPro通过程序化材质建模生成多样化的纹理贴图，支持物理渲染、光照重演等优势。首先利用最新的文生图模型生成多视角参考图像；随后通过基于渲染的优化，结合近期可微分程序化材质进行纹理生成。为此，设计了若干技术解决生成图像与3D网格间的错位问题，并引入新型材质代理，通过部件级理解与对象感知的材质推理提升材质分类与匹配能力。实验表明，该方法在性能上优于现有最先进方法，具备良好的光照重演能力。
原文摘要 · Abstract (English)
In this paper, we present TexPro, a novel method for high-fidelity material generation for input 3D meshes given text prompts. Unlike existing text-conditioned texture generation methods that typically generate RGB textures with baked lighting, TexPro is able to produce diverse texture maps via procedural material modeling, which enables physically-based rendering, relighting, and additional benefits inherent to procedural materials. Specifically, we first generate multi-view reference images given the input textual prompt by employing the latest text-to-image model. We then derive texture maps through rendering-based optimization with recent differentiable procedural materials. To this end, we design several techniques to handle the misalignment between the generated multi-view images and 3D meshes, and introduce a novel material agent that enhances material classification and matching by exploring both part-level understanding and object-aware material reasoning. Experiments demonstrate the superiority of the proposed method over existing SOTAs, and its capability of relighting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。