arXiv:2606.01920cs.CV2026-06

提出分阶段生成与筛选的图像数据蒸馏方法,提升合成数据利用率。

Pool-Select-Refine for Allocation-Aware Generative Dataset Distillation

论文配图:Pool-Select-Refine for Allocation-Aware Generative Dataset Distillation
图 1 · 摘自论文原文
  • 先生成大量候选样本,再按预算精选子集,避免盲目生成
  • 在隐空间用教师模型软标签优化精选样本,提升语义准确性
  • 适合需要高效利用有限数据预算的研究者使用

基于扩散模型的数据蒸馏近期成为将大规模数据集压缩为紧凑合成数据集的有前景范式。通过利用预训练生成先验,这些方法能比传统匹配方法更高效地生成类条件真实样本。然而,多数现有扩散方法仍采用固定的“生成即使用”策略,在固定每类样本数预算下直接将生成样本作为最终蒸馏集。这种设计将候选生成与最终预算分配紧密耦合,可能导致预算浪费或信息量不足。本文提出“Pool-Select-Refine”两阶段框架,实现预算感知的生成式数据蒸馏。首先,构建过完备候选池,再在目标预算下选择紧凑子集;其次,在隐空间使用来自教师模型的软标签对选中样本进行优化,增强语义一致性并保留生成先验。该设计显式解耦生成、选择与优化过程,更高效利用蒸馏预算。在大规模及细粒度图像分类基准上的实验表明,所提框架持续优于扩散基线。结果表明,引入优化前的筛选阶段是提升扩散式数据蒸馏性能的简单而有效方式。

原文摘要 · Abstract (English)

Diffusion-based dataset distillation has recently emerged as a promising paradigm for condensing large-scale datasets into compact synthetic sets. By leveraging pretrained generative priors, these methods can produce realistic class-conditional samples more efficiently than traditional matching-based approaches. However, most existing diffusion-based methods still adopt a rigid ``Generate-and-Use'' strategy, where the generated samples are directly treated as the final distilled set under a fixed images-per-class budget. Such a design tightly couples candidate generation with final budget allocation, which may result in redundant waste of the limited budget or insufficiently informative samples. In this paper, we propose ``Pool-Select-Refine'', a two-stage framework for allocation-aware generative dataset distillation. First, instead of directly using a fixed number of generated samples, we construct an over-complete candidate pool and select a compact subset under the target budget. Second, we refine the selected samples in latent space using soft-label supervision derived from the teacher model, improving semantic alignment while preserving the generative prior. This design explicitly decouples generation, selection, and refinement, enabling more effective use of the distillation budget. Experiments on large-scale and fine-grained image classification benchmarks show that the proposed framework delivers consistent gains over diffusion-based baselines. The results suggest that introducing a curation stage before refinement is a simple yet effective way to improve diffusion-based dataset distillation.

数据蒸馏扩散模型生成式学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。