提升细粒度图文生成精度，让不同类鸟/花的细微差异更真实
Fine-grained Text to Image Synthesis
- 在判别器中加入辅助分类器，增强对细粒度类别的识别能力
- 采用对比学习，使同类图像更相似、异类图像更相异
- 在鸟和花数据集上表现优于现有方法，细节更精准
细粒度图文生成旨在根据描述不同子类别的文本生成图像。与通用图文生成不同，细粒度生成中不同子类图像高度相似，且同一图像可能对应语言表述不同的文本。尽管近期生成对抗网络（GAN），如循环仿射变换（RAT）GAN模型，已能生成清晰逼真的图像，但其忽略细粒度信息。本文提出在判别器中引入辅助分类器，并结合对比学习方法，以提升RAT GAN合成图像的细粒度准确性。辅助分类器帮助判别器区分图像类别，促使生成器生成更精确的细粒度内容；对比学习则最小化不同子类间图像相似度，最大化同子类内图像相似度。我们在常用CUB-200-2011鸟类数据集和Oxford-102花卉数据集上评估多种前沿方法，结果表明本方法性能更优。
原文摘要 · Abstract (English)
Fine-grained text to image synthesis involves generating images from texts that belong to different categories. In contrast to general text to image synthesis, in fine-grained synthesis there is high similarity between images of different subclasses, and there may be linguistic discrepancy among texts describing the same image. Recent Generative Adversarial Networks (GAN), such as the Recurrent Affine Transformation (RAT) GAN model, are able to synthesize clear and realistic images from texts. However, GAN models ignore fine-grained level information. In this paper we propose an approach that incorporates an auxiliary classifier in the discriminator and a contrastive learning method to improve the accuracy of fine-grained details in images synthesized by RAT GAN. The auxiliary classifier helps the discriminator classify the class of images, and helps the generator synthesize more accurate fine-grained images. The contrastive learning method minimizes the similarity between images from different subclasses and maximizes the similarity between images from the same subclass. We evaluate on several state-of-the-art methods on the commonly used CUB-200-2011 bird dataset and Oxford-102 flower dataset, and demonstrated superior performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。