通过消除标签不一致并优化学习模式，提升数据蒸馏性能。
Enhancing Dataset Distillation via Label Inconsistency Elimination and Learning Pattern Refinement
- 移除软标签以解决标签对应不一致问题
- 缩小匹配范围，聚焦易学模式提升效果
- 在CIFAR-100和Tiny ImageNet上排名榜首
数据蒸馏（Dataset Distillation, DD）旨在生成一个精简数据集，使模型在该数据集上训练后能达到与在原始全量数据上训练相当的性能，从而降低计算资源、存储和时间成本。本文提出在ECCV-2024数据蒸馏挑战赛（第1赛道）中排名第一的方案——改进的难度对齐轨迹匹配（M-DATM）。该方法对现有最先进方法DATM进行两处关键修改：（1）原方法学习的软标签与官方评估脚本生成的标签无法一一对应，因此我们移除了软标签机制以缓解不一致性；（2）移除软标签后，合成数据难以捕捉后期轨迹信息，尤其在Tiny ImageNet上表现受限，故我们缩减匹配范围，使合成数据更集中于简单模式的学习。最终评估中，M-DATM在CIFAR-100和Tiny ImageNet上的准确率分别为0.4061和0.1831，在固定每类图像数（IPC）赛道排名第一。
原文摘要 · Abstract (English)
Dataset Distillation (DD) seeks to create a condensed dataset that, when used to train a model, enables the model to achieve performance similar to that of a model trained on the entire original dataset. It relieves the model training from processing massive data and thus reduces the computation resources, storage, and time costs. This paper illustrates our solution that ranks 1st in the ECCV-2024 Data Distillation Challenge (track 1). Our solution, Modified Difficulty-Aligned Trajectory Matching (M-DATM), introduces two key modifications to the original state-of-the-art method DATM: (1) the soft labels learned by DATM do not achieve one-to-one correspondence with the counterparts generated by the official evaluation script, so we remove the soft labels technique to alleviate such inconsistency; (2) since the removal of soft labels makes it harder for the synthetic dataset to learn late trajectory information, particularly on Tiny ImageNet, we reduce the matching range, allowing the synthetic data to concentrate more on the easier patterns. In the final evaluation, our M-DATM achieved accuracies of 0.4061 and 0.1831 on the CIFAR-100 and Tiny ImageNet datasets, ranking 1st in the Fixed Images Per Class (IPC) Track.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。