用生成图像模型预测3D场景材质，快速构建一致的纹理贴图
An evaluation of SVBRDF Prediction from Generative Image Models for Appearance Modeling of 3D Scenes
- 用生成图像模型输入，通过UNet网络预测多视图材质参数
- 单视角预测易不一致，但生成图像提供额外信息提升准确性
- 简单UNet表现媲美复杂结构，适合快速建模应用
深度生成模型正深刻改变数字内容创作。在纹理生成方面，条件图像生成器可合成与3D几何一致的逼真RGB图像；在外观建模方面，SVBRDF预测网络可从RGB图像恢复材质参数。结合二者，可快速为3D场景多个视角生成SVBRDF贴图并合并为纹理贴图集。本文分析此类快速外观建模流程中SVBRDF预测的挑战与机遇。一方面，单视角预测可能导致多视角不一致，影响贴图一致性；另一方面，生成图像及其不同条件（如文本、草图）可提供比照片更丰富的材质信息。我们对比了多种神经架构与条件设置，发现标准UNet在精度与一致性上与复杂设计相当，具有实用价值。
原文摘要 · Abstract (English)
Digital content creation is experiencing a profound change with the advent of deep generative models. For texturing, conditional image generators now allow the synthesis of realistic RGB images of a 3D scene that align with the geometry of that scene. For appearance modeling, SVBRDF prediction networks recover material parameters from RGB images. Combining these technologies allows us to quickly generate SVBRDF maps for multiple views of a 3D scene, which can be merged to form a SVBRDF texture atlas of that scene. In this paper, we analyze the challenges and opportunities for SVBRDF prediction in the context of such a fast appearance modeling pipeline. On the one hand, single-view SVBRDF predictions might suffer from multiview incoherence and yield inconsistent texture atlases. On the other hand, generated RGB images, and the different modalities on which they are conditioned, can provide additional information for SVBRDF estimation compared to photographs. We compare neural architectures and conditions to identify designs that achieve high accuracy and coherence. We find that, surprisingly, a standard UNet is competitive with more complex designs. Project page: http://repo-sam.inria.fr/nerphys/svbrdf-evaluation
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。