用扩散模型无须微调即可高效生成多样化数据集，提升训练效果。
MGD$^3$: Mode-Guided Dataset Distillation using Diffusion Models
- 基于预训练扩散模型，分三阶段引导模式发现与多样性增强
- 在ImageNet系列数据集上最高提升4.4%准确率，且无需微调
- 适合追求低计算成本与高生成多样性的数据压缩研究者
数据集蒸馏已成为降低训练成本、提升模型部署效率的有效策略。近期方法利用生成模型捕捉数据分布进行蒸馏，但通常需通过蒸馏损失对模型进行微调以保证样本多样性，而现有方法无法确保多样性，制约性能。本文提出一种无需微调的模式引导扩散模型：通过三阶段机制实现——模式发现以识别不同数据模式，模式引导增强类内多样性，停止引导缓解合成样本中的伪影问题。实验表明，该方法在ImageNette、ImageIDC、ImageNet-100和ImageNet-1K上分别取得4.4%、2.9%、1.6%和1.6%的准确率提升，显著优于当前最优方法。同时避免了蒸馏损失微调，大幅降低计算开销。代码已公开于项目主页。
原文摘要 · Abstract (English)
Dataset distillation has emerged as an effective strategy, significantly reducing training costs and facilitating more efficient model deployment. Recent advances have leveraged generative models to distill datasets by capturing the underlying data distribution. Unfortunately, existing methods require model fine-tuning with distillation losses to encourage diversity and representativeness. However, these methods do not guarantee sample diversity, limiting their performance. We propose a mode-guided diffusion model leveraging a pre-trained diffusion model without the need to fine-tune with distillation losses. Our approach addresses dataset diversity in three stages: Mode Discovery to identify distinct data modes, Mode Guidance to enhance intra-class diversity, and Stop Guidance to mitigate artifacts in synthetic samples that affect performance. Our approach outperforms state-of-the-art methods, achieving accuracy gains of 4.4%, 2.9%, 1.6%, and 1.6% on ImageNette, ImageIDC, ImageNet-100, and ImageNet-1K, respectively. Our method eliminates the need for fine-tuning diffusion models with distillation losses, significantly reducing computational costs. Our code is available on the project webpage: https://jachansantiago.github.io/mode-guided-distillation/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。