用扩散模型提升3D贴图分辨率,让老旧资产重获新生
Texture++: Elevating 3D Asset Texture Resolution with a Region-Aware Diffusion Model

- 将贴图超分转为多视角生成再融合,提升连续性
- 通过四叉树组织区域并生成掩码,精准定位需增强部分
- 基于扩散模型的局部增强,细节更丰富且无缝衔接
大量3D资产因纹理分辨率低而被废弃,现有超分辨率模型忽略贴图,仅关注自然图像。我们提出Texture++,一种新型纹理超分辨率框架,可将低分辨率贴图转化为高分辨率高质量结果。具体而言,将UV空间中的超分任务重构为多视角渲染后融合。首先,提出自适应视图选择策略,整合分散在不同UV贴图块中的纹理信息;其次,设计基于四叉树的纹理区域组织方法,对不同视角的超分结果进行合并,并生成掩码以区分需优化区域;最后,构建基于扩散模型的超分辨率模块,针对指定掩码区域增强细节,并与周围区域无缝融合。综合评估表明,该方法在细节和连贯性上显著优于现有技术。
原文摘要 · Abstract (English)
Numerous 3D assets are discarded due to low texture resolution, while current super-resolution models ignore texture maps and focus on natural images. An efficient and generalizable texture super-resolution model can revitalize a large corpus of aging yet valuable assets across industries such as film and video games. We present Texture++, a novel framework for texture super-resolution, which enhances the low-resolution textures of assets to produce high-resolution, high-quality results. Specifically, we reformulate the task of super-resolution in UV space into performing it across multiple rendered views and merging the outputs. Firstly, to achieve more complete and continuous textures in the view space, we propose an adaptive view selection strategy to integrate textures dispersed across UV texture patches. Furthermore, we introduce a quadtree-based texture region organization method for combining super-resolved textures from different viewpoints, providing masks to distinguish regions that require improvement. Finally, we design a diffusion-based super-resolution model that enhances the texture resolution for specified masked regions, seamlessly integrating with surrounding regions. Through comprehensive evaluations, we demonstrate that our approach yields textures with substantially improved detail and coherence over existing methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。