用扩散模型直接生成高分辨率3D纹理，支持文本和单图驱动
TEXGen: a Generative Diffusion Model for Mesh Textures
- 设计新型网络结构，融合UV贴图卷积与点云注意力
- 训练7亿参数模型，可直接生成高分辨率纹理贴图
- 支持文本修复、稀疏视图补全等扩展应用
高质量纹理贴图对真实感3D资产渲染至关重要，但现有研究较少在大规模数据集上直接学习纹理空间。本文摒弃依赖预训练2D扩散模型进行3D纹理测试时优化的传统方法，转而聚焦于在UV纹理空间中直接学习。首次训练了一个大规模扩散模型，可前馈生成高分辨率纹理贴图。为提升高分辨率UV空间中的学习效率，提出一种可扩展的网络架构，将卷积操作应用于UV贴图，同时在点云上引入注意力机制。基于该设计，训练了一个7亿参数的扩散模型，能够根据文本提示或单张图像生成对应的UV纹理贴图。模型训练完成后，天然支持多种扩展应用，包括文本引导的纹理修复、稀疏视图纹理补全及文本驱动的纹理合成。项目页面见http://cvmi-lab.github.io/TEXGen/
原文摘要 · Abstract (English)
While high-quality texture maps are essential for realistic 3D asset rendering, few studies have explored learning directly in the texture space, especially on large-scale datasets. In this work, we depart from the conventional approach of relying on pre-trained 2D diffusion models for test-time optimization of 3D textures. Instead, we focus on the fundamental problem of learning in the UV texture space itself. For the first time, we train a large diffusion model capable of directly generating high-resolution texture maps in a feed-forward manner. To facilitate efficient learning in high-resolution UV spaces, we propose a scalable network architecture that interleaves convolutions on UV maps with attention layers on point clouds. Leveraging this architectural design, we train a 700 million parameter diffusion model that can generate UV texture maps guided by text prompts and single-view images. Once trained, our model naturally supports various extended applications, including text-guided texture inpainting, sparse-view texture completion, and text-driven texture synthesis. Project page is at http://cvmi-lab.github.io/TEXGen/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。