用扩散模型生成可信图像，混合真实与合成数据提升视觉模型性能。
Context-guided Responsible Data Augmentation with Diffusion Models
- 基于上下文和负提示控制扩散过程，生成可靠增强样本。
- 在多个数据集上提升准确率，最高达3%绝对增长。
- 适合需要数据增强的视觉模型训练者使用。
生成式扩散模型为训练复杂视觉模型提供了自然的数据增强选择。然而，确保其生成内容作为增强样本的可靠性仍是一个开放挑战。尽管已有多种方法利用生成图像强化模型训练，但如何有效结合自然图像与生成图像作为丰富的监督信号尚不明确。为此，我们提出一种文本到图像数据增强方法 DiffCoRe-Mix，通过显式约束的扩散模型，利用样本上下文和负提示生成可靠的增强样本。为保留关键语义特征，我们在 CLIP 嵌入空间中引入硬余弦过滤机制，剔除不理想生成样本。该方法系统地在像素和块级别混合真实与生成图像。我们在 ImageNet-1K、Tiny ImageNet-200、CIFAR-100、Flowers102、CUB-Birds、Stanford Cars 和 Caltech 等数据集上进行广泛评估，结果表明性能全面提升，相较于现有最佳方法，顶级准确率最高提升约3%，同时计算开销相当。代码已公开于 https://github.com/khawar-islam/DiffCoRe-Mix。
原文摘要 · Abstract (English)
Generative diffusion models offer a natural choice for data augmentation when training complex vision models. However, ensuring reliability of their generative content as augmentation samples remains an open challenge. Despite a number of techniques utilizing generative images to strengthen model training, it remains unclear how to utilize the combination of natural and generative images as a rich supervisory signal for effective model induction. In this regard, we propose a text-to-image (T2I) data augmentation method, named DiffCoRe-Mix, that computes a set of generative counterparts for a training sample with an explicitly constrained diffusion model that leverages sample-based context and negative prompting for a reliable augmentation sample generation. To preserve key semantic axes, we also filter out undesired generative samples in our augmentation process. To that end, we propose a hard-cosine filtration in the embedding space of CLIP. Our approach systematically mixes the natural and generative images at pixel and patch levels. We extensively evaluate our technique on ImageNet-1K,Tiny ImageNet-200, CIFAR-100, Flowers102, CUB-Birds, Stanford Cars, and Caltech datasets, demonstrating a notable increase in performance across the board, achieving up to $\sim 3\%$ absolute gain for top-1 accuracy over the state-of-the-art methods, while showing comparable computational overhead. Our code is publicly available at https://github.com/khawar-islam/DiffCoRe-Mix
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。