arXiv:2512.17302cs.CV2025-12被引 1

通过微调VAE构建材质隐空间,生成高质量PBR贴图。

MatLat: Material Latent Space for PBR Texture Generation

  • 微调预训练VAE以保持材质通道的隐空间一致性。
  • 引入局部性正则化,提升跨视角纹理一致性。
  • 在真实3D网格上生成高保真PBR贴图,优于现有方法。

我们提出一种生成框架,用于在给定3D网格上生成高质量PBR贴图。由于大规模PBR纹理数据集稀缺,本方法聚焦于有效利用预训练隐空间和扩散先验,通过定向微调学习一个材质隐空间MatLat。与以往冻结嵌入网络导致编码新增PBR通道时分布偏移、阻碍后续扩散训练的方法不同,我们微调预训练VAE,使新材质通道可低偏差融入。进一步表明,仅靠对应注意力不足以保证跨视角一致性,除非隐空间到图像的映射保持局部性。为此,我们在VAE微调中引入正则化:裁剪隐空间块,解码后对齐对应图像区域,强化像素-隐空间的空间对应关系。消融实验与基线对比显示,该框架显著提升PBR贴图保真度,各组件对达到顶尖性能至关重要。

原文摘要 · Abstract (English)

We propose a generative framework for producing high-quality PBR textures on a given 3D mesh. As large-scale PBR texture datasets are scarce, our approach focuses on effectively leveraging the embedding space and diffusion priors of pretrained latent image generative models while learning a material latent space, MatLat, through targeted fine-tuning. Unlike prior methods that freeze the embedding network and thus lead to distribution shifts when encoding additional PBR channels and hinder subsequent diffusion training, we fine-tune the pretrained VAE so that new material channels can be incorporated with minimal latent distribution deviation. We further show that correspondence-aware attention alone is insufficient for cross-view consistency unless the latent-to-image mapping preserves locality. To enforce this locality, we introduce a regularization in the VAE fine-tuning that crops latent patches, decodes them, and aligns the corresponding image regions to maintain strong pixel-latent spatial correspondence. Ablation studies and comparison with previous baselines demonstrate that our framework improves PBR texture fidelity and that each component is critical for achieving state-of-the-art performance.

PBR生成隐空间扩散模型材质生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。