用线性外推扩充小数据集，提升文本生成图像质量
Data Extrapolation for Text-to-image Generation on Small Datasets
- 仅对文本特征做线性外推，通过搜索引擎获取新图像
- 构建的数据量达原数据数十倍，CUB等数据集上FID降至5.00
- 适合资源有限但需高质量图像生成的研究者
文本到图像生成需要大量训练数据以生成高质量图像。以往方法依赖裁剪、翻转、混合等数据插值手段，无法引入新信息，增益有限。本文提出一种基于线性外推的新数据增强方法，仅对文本特征进行外推，并通过搜索引擎检索对应图像。为保证新图文对可靠性，设计双重异常检测器净化检索结果。基于此方法，训练样本规模扩大数十倍，显著提升生成性能。此外，提出NULL-guidance优化得分估计，并使用递归仿射变换融合文本信息。模型在CUB、Oxford、COCO数据集上分别取得7.91、9.52、5.00的FID分数。代码与数据将公开于GitHub（https://github.com/senmaoy/RAT-Diffusion）。
原文摘要 · Abstract (English)
Text-to-image generation requires large amount of training data to synthesizing high-quality images. For augmenting training data, previous methods rely on data interpolations like cropping, flipping, and mixing up, which fail to introduce new information and yield only marginal improvements. In this paper, we propose a new data augmentation method for text-to-image generation using linear extrapolation. Specifically, we apply linear extrapolation only on text feature, and new image data are retrieved from the internet by search engines. For the reliability of new text-image pairs, we design two outlier detectors to purify retrieved images. Based on extrapolation, we construct training samples dozens of times larger than the original dataset, resulting in a significant improvement in text-to-image performance. Moreover, we propose a NULL-guidance to refine score estimation, and apply recurrent affine transformation to fuse text information. Our model achieves FID scores of 7.91, 9.52 and 5.00 on the CUB, Oxford and COCO datasets. The code and data will be available on GitHub (https://github.com/senmaoy/RAT-Diffusion).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。