arXiv:2412.04668cs.CVcs.AI2024-12被引 2

用生成模型动态扩展核心数据集,实现高效高质压缩。

Diffusion-Augmented Coreset Expansion for Scalable Dataset Distillation

  • 先选关键图像块组成核心集,再用生成模型实时增强分辨率与多样性。
  • 在多个大规模数据蒸馏任务上性能超越当前最优超10%。
  • 适合需要高效数据压缩与高质量合成数据的场景。

随着神经网络规模迅速扩大,数据存储与传输需求日益增加。数据蒸馏作为一种有前景的解决方案,通过求解双层优化问题,将海量数据中的信息压缩为少量合成样本。然而,现有方法在高分辨率数据和复杂架构下面临计算效率挑战。近期基于知识蒸馏的数据压缩方法提升了可行性。随着生成式基础模型的发展,现在有机会实现更高压缩率、提升蒸馏数据质量并引入数据表示多样性。本文提出两阶段方案:首先通过选择最具信息量的图像块构建核心集(coreset);其次利用生成式基础模型实时动态扩展该核心集,提升图像块分辨率并引入可控变异。大量实验表明,该方法在多种数据蒸馏基准上具备鲁棒性与高效性,在多个大规模任务中相比当前最优方法性能提升超过10%。代码即将开源。

原文摘要 · Abstract (English)

With the rapid scaling of neural networks, data storage and communication demands have intensified. Dataset distillation has emerged as a promising solution, condensing information from extensive datasets into a compact set of synthetic samples by solving a bilevel optimization problem. However, current methods face challenges in computational efficiency, particularly with high-resolution data and complex architectures. Recently, knowledge-distillation-based dataset condensation approaches have made this process more computationally feasible. Yet, with the recent developments of generative foundation models, there is now an opportunity to achieve even greater compression, enhance the quality of distilled data, and introduce valuable diversity into the data representation. In this work, we propose a two-stage solution. First, we compress the dataset by selecting only the most informative patches to form a coreset. Next, we leverage a generative foundation model to dynamically expand this compressed set in real-time, enhancing the resolution of these patches and introducing controlled variability to the coreset. Our extensive experiments demonstrate the robustness and efficiency of our approach across a range of dataset distillation benchmarks. We demonstrate a significant improvement of over 10% compared to the state-of-the-art on several large-scale dataset distillation benchmarks. The code will be released soon.

数据蒸馏生成模型核心集高效压缩

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。