用视觉变压器融合空间生成对抗网络，提升纹理合成质量。
Texture Image Synthesis Using Spatial GAN Based on Vision Transformers
- 将均值方差与文本块特征融入视觉变压器自注意力机制
- 在FID、IS、SSIM和LPIPS上优于当前最优模型
- 适合需要高保真纹理生成的图形与仿真应用
纹理合成是计算机视觉中的基础任务，旨在为从图形到科学模拟的广泛应用生成视觉逼真且结构一致的纹理。传统方法如拼接与基于块的技术在处理复杂纹理时表现不佳，而深度学习的进步已显著推动该领域发展。本文提出ViT-SGAN，一种融合视觉变压器（ViTs）与空间生成对抗网络（SGAN）的新型混合模型，以克服先前方法的局限性。通过在ViTs的自注意力机制中引入均值-方差（mu, sigma）及文本块（textons）等专用纹理描述符，模型增强了对复杂空间依赖关系的捕捉能力，显著提升了纹理质量，尤其在规则与不规则纹理上表现更优。在FID、IS、SSIM和LPIPS等指标上的对比实验验证了其优越性，证明该方法在生成多样化真实纹理方面具有高效性。
原文摘要 · Abstract (English)
Texture synthesis is a fundamental task in computer vision, whose goal is to generate visually realistic and structurally coherent textures for a wide range of applications, from graphics to scientific simulations. While traditional methods like tiling and patch-based techniques often struggle with complex textures, recent advancements in deep learning have transformed this field. In this paper, we propose ViT-SGAN, a new hybrid model that fuses Vision Transformers (ViTs) with a Spatial Generative Adversarial Network (SGAN) to address the limitations of previous methods. By incorporating specialized texture descriptors such as mean-variance (mu, sigma) and textons into the self-attention mechanism of ViTs, our model achieves superior texture synthesis. This approach enhances the model's capacity to capture complex spatial dependencies, leading to improved texture quality that is superior to state-of-the-art models, especially for regular and irregular textures. Comparison experiments with metrics such as FID, IS, SSIM, and LPIPS demonstrate the substantial improvement of ViT-SGAN, which underlines its efficiency in generating diverse realistic textures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。