用注意力聚焦关键特征,让精简数据集更精准、泛化更强。
Dataset Distillation Based on Saliency-Driven Prototype Alignment

- 基于梯度显著性生成类判别原型,突出核心特征区域。
- 在多个基准上超越强基线,小数据集也能保持高精度。
- 仅训练轻量分类器,兼容主流扩散模型,适合资源受限场景。
数据蒸馏旨在合成紧凑数据集,在显著降低计算与存储成本的同时逼近全量数据训练性能。然而,基于扩散的方法常因潜在原型与类判别区域对齐不足、受无关背景干扰,导致结构一致性与泛化能力下降。为此,提出一种显著性驱动的蒸馏框架:第一阶段,利用集成Grad-CAM++显著性构建强调类判别区域的原型;第二阶段,通过硬原型精炼生成具有挑战性且类别一致的原型,提升判别力与多样性。重要的是,扩散主干(如LDM和DiT)保持冻结,仅训练轻量级分类器用于显著性提取。跨多个基准的大量实验表明,该方法持续优于强基线。代码将公开。
原文摘要 · Abstract (English)
Dataset distillation aims to synthesize compact datasets that can approximate the performance of full-data training while significantly reducing computational and storage costs. However, diffusion-based distillation methods often struggle to preserve structural coherence and generalization, especially in visually complex domains. This issue often stems from latent prototypes that are weakly aligned with class-discriminative regions and contaminated by irrelevant background, thereby degrading generation quality and generalization. To address this limitation, we propose a saliency-driven distillation framework that constructs class-discriminative latent prototypes to enhance representativeness and generalization. The framework proceeds in two stages: (1) ensemble Grad-CAM++ saliency is used to construct prototypes emphasizing class-discriminative regions, and (2) hard-prototype refinement is then applied to construct challenging yet class-consistent prototypes, thereby enhancing discriminability and diversity. Importantly, the diffusion backbones (e.g., LDM and DiT) remain frozen; only lightweight classifiers used for saliency extraction are trained. Extensive experiments across multiple benchmarks demonstrate consistent performance improvements over strong baselines. Code will be released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。