用大模型从图片生成程序化材质，省去专业操作。
VLMaterial: Procedural Material Generation with Large Vision-Language Models
- 将材质转为Python代码，用视觉语言模型生成对应代码。
- 在真实和合成图像上均优于现有方法，生成效果更逼真。
- 开源数据集+代码增强，适合图形设计与自动化工具开发者。
程序化材质以功能节点图形式存在，广泛用于计算机图形学中实现逼真材质外观设计，支持用户直观精确地编辑以达到期望视觉效果。然而，根据输入图像创建程序化材质需要专业知识且耗时费力。本文利用将程序化材质转换为标准Python程序的能力，并微调一个大规模预训练视觉语言模型（VLM），从输入图像生成此类程序。为支持有效微调，我们还贡献了一个开源的程序化材质数据集，并提出通过另一个预训练大型语言模型（LLM）提示实现程序级数据增强。通过大量评估，我们证明该方法在合成与真实世界示例上均优于先前方法。
原文摘要 · Abstract (English)
Procedural materials, represented as functional node graphs, are ubiquitous in computer graphics for photorealistic material appearance design. They allow users to perform intuitive and precise editing to achieve desired visual appearances. However, creating a procedural material given an input image requires professional knowledge and significant effort. In this work, we leverage the ability to convert procedural materials into standard Python programs and fine-tune a large pre-trained vision-language model (VLM) to generate such programs from input images. To enable effective fine-tuning, we also contribute an open-source procedural material dataset and propose to perform program-level augmentation by prompting another pre-trained large language model (LLM). Through extensive evaluation, we show that our method outperforms previous methods on both synthetic and real-world examples.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。