提出新方法提升长尾数据集蒸馏效果，稳定且更真实。
Towards Principled Dataset Distillation: A Spectral Distribution Perspective
- 用谱分布匹配替代传统距离度量，更好对齐数据分布。
- 在CIFAR-10-LT上，每类仅10张图时性能领先当前方法14.0%。
- 自适应增强尾部类别真实性，适合处理数据不平衡场景。
数据集蒸馏（DD）旨在将大规模数据集压缩为紧凑的合成数据以实现高效训练。然而，现有方法在长尾数据集上表现显著下降。我们识别出两大根本问题：分布差异度量的启发式设计、对不平衡类别的均等处理。为此，提出类别感知谱分布匹配（CSDM），通过优良核函数的谱重构分布对齐。该方法将原始样本映射到频域，得到谱分布距离（SDD）。为缓解类别不平衡，利用SDD的统一形式进行幅值-相位分解，自适应强化尾部类别的真实性。在CIFAR-10-LT数据集上，每类仅10张图像时，相比最优已有方法提升14.0%；当尾部类别图像数从500降至25时，性能仅下降5.7%，展现出强鲁棒性。
原文摘要 · Abstract (English)
Dataset distillation (DD) aims to compress large-scale datasets into compact synthetic counterparts for efficient model training. However, existing DD methods exhibit substantial performance degradation on long-tailed datasets. We identify two fundamental challenges: heuristic design choices for distribution discrepancy measure and uniform treatment of imbalanced classes. To address these limitations, we propose Class-Aware Spectral Distribution Matching (CSDM), which reformulates distribution alignment via the spectrum of a well-behaved kernel function. This technique maps the original samples into frequency space, resulting in the Spectral Distribution Distance (SDD). To mitigate class imbalance, we exploit the unified form of SDD to perform amplitude-phase decomposition, which adaptively prioritizes the realism in tail classes. On CIFAR-10-LT, with 10 images per class, CSDM achieves a 14.0% improvement over state-of-the-art DD methods, with only a 5.7% performance drop when the number of images in tail classes decreases from 500 to 25, demonstrating strong stability on long-tailed data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。