用扩散模型高效生成带标签的合成图像，大幅降低数据存储与训练成本。
Data-Efficient Generation for Dataset Distillation
- 训练条件扩散模型生成类标签合成图像，保持真实感与可读性。
- 每秒生成数十张图像，合成数据仅需少量即可支持下游任务训练。
- 在CIFAR100和TinyImageNet上获ECCV2024首届数据蒸馏挑战赛第一名。
深度学习在图像任务中表现优异，但数据存储与计算成本呈指数增长，成为主要挑战。数据蒸馏通过为每类生成少量合成图像来解决此问题，这些图像包含全部关键信息。现有方法多聚焦于匹配，但合成图像往往不可读，且下游任务性能不足；当每类合成图像数量略有增加时，蒸馏时间迅速失控。为此，本文训练了一个类条件潜空间扩散模型，可生成带有标签的逼真合成图像，采样速度可达每秒数十张。我们证明，仅使用少量合成图像即可有效训练模型，并在大规模真实测试集上评估。该方法在ECCV 2024首届数据蒸馏挑战赛中，于CIFAR100与TinyImageNet数据集上取得排名第一的成绩。
原文摘要 · Abstract (English)
While deep learning techniques have proven successful in image-related tasks, the exponentially increased data storage and computation costs become a significant challenge. Dataset distillation addresses these challenges by synthesizing only a few images for each class that encapsulate all essential information. Most current methods focus on matching. The problems lie in the synthetic images not being human-readable and the dataset performance being insufficient for downstream learning tasks. Moreover, the distillation time can quickly get out of bounds when the number of synthetic images per class increases even slightly. To address this, we train a class conditional latent diffusion model capable of generating realistic synthetic images with labels. The sampling time can be reduced to several tens of images per seconds. We demonstrate that models can be effectively trained using only a small set of synthetic images and evaluated on a large real test set. Our approach achieved rank \(1\) in The First Dataset Distillation Challenge at ECCV 2024 on the CIFAR100 and TinyImageNet datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。