用低秩空间采样压缩数据集，提升训练效率与模型性能。
Efficient Dataset Distillation through Low-Rank Space Sampling
- 通过低秩近似捕捉数据多维流形子空间，生成合成数据。
- 在CIFAR-10等数据集上平均提升9.9%性能，优于基线方法。
- 适合需要高效数据压缩与高泛化能力的场景。
大规模数据是深度学习成功的关键，但冗余信息会削弱模型泛化能力并增加计算负担。数据集蒸馏（DD）将原始数据集压缩为更小但具代表性的子集，以实现高质量数据和高效训练策略。现有方法将每张图像视为独立实体生成合成图像，忽略了数据间的共性特征。本文提出基于低秩空间采样匹配训练轨迹（MTT-LSS）的数据集蒸馏方法，利用低秩近似捕获原始数据的多个低维流形子空间。合成数据由这些子空间的基础向量与共享维度映射器表示，降低生成单个数据点的成本，同时有效减少信息冗余。该方法在CIFAR-10、CIFAR-100和SVHN数据集上测试，平均性能优于基线方法9.9%。
原文摘要 · Abstract (English)
Huge amount of data is the key of the success of deep learning, however, redundant information impairs the generalization ability of the model and increases the burden of calculation. Dataset Distillation (DD) compresses the original dataset into a smaller but representative subset for high-quality data and efficient training strategies. Existing works for DD generate synthetic images by treating each image as an independent entity, thereby overlooking the common features among data. This paper proposes a dataset distillation method based on Matching Training Trajectories with Low-rank Space Sampling(MTT-LSS), which uses low-rank approximations to capture multiple low-dimensional manifold subspaces of the original data. The synthetic data is represented by basis vectors and shared dimension mappers from these subspaces, reducing the cost of generating individual data points while effectively minimizing information redundancy. The proposed method is tested on CIFAR-10, CIFAR-100, and SVHN datasets, and outperforms the baseline methods by an average of 9.9%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。