通过类内分组降低图像相似性，实现软标签40倍压缩
Are Large-scale Soft Labels Necessary for Large-scale Dataset Distillation?
- 类内分批生成图像，减少类内相似性
- 软标签体积从113GB压缩至2.8GB（40倍）
- 适合关注数据蒸馏存储优化的研究者
在ImageNet-condensation中，辅助软标签的存储量超过压缩数据集的30倍。本文首次发现，压缩数据集中的高类内相似性需要大规模软标签。这源于以往方法在批量归一化匹配时混合不同类别的样本。为降低类内相似性，我们引入类内监督，通过类内分批生成图像。结果提升了类内多样性，大幅减少所需软标签规模。关键优势是：图像多样性提升后，可通过简单随机剪枝实现软标签压缩，无需复杂规则策略。实验验证：将ImageNet-1K压缩至每类200张图像时，软标签由113GB降至2.8GB（40倍压缩），性能提升2.6%。代码已开源。
原文摘要 · Abstract (English)
In ImageNet-condensation, the storage for auxiliary soft labels exceeds that of the condensed dataset by over 30 times. However, are large-scale soft labels necessary for large-scale dataset distillation? In this paper, we first discover that the high within-class similarity in condensed datasets necessitates the use of large-scale soft labels. This high within-class similarity can be attributed to the fact that previous methods use samples from different classes to construct a single batch for batch normalization (BN) matching. To reduce the within-class similarity, we introduce class-wise supervision during the image synthesizing process by batching the samples within classes, instead of across classes. As a result, we can increase within-class diversity and reduce the size of required soft labels. A key benefit of improved image diversity is that soft label compression can be achieved through simple random pruning, eliminating the need for complex rule-based strategies. Experiments validate our discoveries. For example, when condensing ImageNet-1K to 200 images per class, our approach compresses the required soft labels from 113 GB to 2.8 GB (40x compression) with a 2.6% performance gain. Code is available at: https://github.com/he-y/soft-label-pruning-for-dataset-distillation
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。