arXiv:2605.03877cs.CVcs.AI2026-05中稿 · CVPR被引 4

无需微调的扩散模型数据蒸馏,提升生成数据质量。

DMGD: Train-Free Dataset Distillation with Semantic-Distribution Matching in Diffusion Models

论文配图:DMGD: Train-Free Dataset Distillation with Semantic-Distribution Matching in Diffusion Models
图 1 · 摘自论文原文
  • 通过语义匹配与分布对齐实现无训练引导
  • 在ImageNet系列数据集上平均提升2.1%-5.4%准确率
  • 适合追求高效、免微调数据蒸馏的研究者

数据蒸馏通过将大规模数据集的信息浓缩为更小的合成数据集,实现高效训练。近年来基于扩散模型的方法为数据蒸馏提供了新思路,但通常需要额外的微调阶段,且有效引导机制仍不充分。为此,我们重新思考基于扩散模型的数据蒸馏,提出双匹配引导扩散(DMGD)框架,核心是高效的无训练引导。首先通过条件似然优化建立语义匹配,无需辅助分类器;其次提出动态引导机制,在保持语义一致性的同时增强合成数据多样性。同时引入基于最优传输(OT)的分布匹配方法,进一步对齐目标分布结构。为保证效率,设计两种增强策略:分布近似匹配与贪婪渐进匹配,实现低计算开销下的有效分布引导。在ImageNet-Woof、ImageNet-Nette和ImageNet-1K上的实验表明,该无训练方法显著优于需额外微调的最先进方法,平均准确率分别提升2.1%、5.4%和2.4%。

原文摘要 · Abstract (English)

Dataset distillation enables efficient training by distilling the information of large-scale datasets into significantly smaller synthetic datasets. Diffusion based paradigms have emerged in recent years, offering novel perspectives for dataset distillation. However, they typically necessitate additional fine-tuning stages, and effective guidance mechanisms remain underexplored. To address these limitations, we rethink diffusion based dataset distillation and propose a Dual Matching Guided Diffusion (DMGD) framework, centered on efficient training-free guidance. We first establish Semantic Matching via conditional likelihood optimization, eliminating the need for auxiliary classifiers. Furthermore, we propose a dynamic guidance mechanism that enhances the diversity of synthetic data while maintaining semantic alignment. Simultaneously, we introduce an optimal transport (OT) based Distribution Matching approach to further align with the target distribution structure. To ensure efficiency, we develop two enhanced strategies for diffusion based framework: Distribution Approximate Matching and Greedy Progressive Matching. These strategies enable effective distribution matching guidance with minimal computational overhead. Experimental results on ImageNet-Woof, ImageNet-Nette, and ImageNet-1K demonstrate that our training-free approach achieves significant improvements, outperforming state-of-the-art (SOTA) methods requiring additional fine-tuning by average accuracy gains of 2.1%, 5.4%, and 2.4%, respectively.

数据蒸馏扩散模型无训练分布对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。