通过解耦模型先验，生成更具类内多样性的压缩数据集。
PRISM: Diversifying Dataset Distillation by Decoupling Architectural Priors
- 用不同教师模型分别指导逻辑匹配与归一化对齐
- 在低中等压缩比下超越单教师与多教师方法
- 适合需要高多样性合成数据的场景
数据集蒸馏（DD）旨在生成紧凑而忠实的合成数据，但现有方法常继承单一教师模型的归纳偏置。随着数据规模增大，这种偏置导致生成样本过于平滑且同质，降低类内多样性并限制泛化能力。我们提出PRISM（来自多样化源模型的先验），在合成过程中解耦架构先验。PRISM将逻辑匹配与正则化目标分离，分别由主教师模型和随机子集教师模型监督：前者负责输出逻辑匹配，后者用于批归一化（BN）对齐。在ImageNet-1K上，PRISM在低-中等每类图像数（IPC）条件下持续且可复现地优于单教师方法（如SRe2L）及近期多教师变体（如G-VBSM）。生成数据展现出显著更高的类内多样性，表现为特征间余弦相似度明显下降。我们还分析了教师选择策略（预训练与蒸馏中选择），并引入可扩展的跨类批处理方案，实现快速并行合成。代码将在审稿期后发布。
原文摘要 · Abstract (English)
Dataset distillation (DD) promises compact yet faithful synthetic data, but existing approaches often inherit the inductive bias of a single teacher model. As dataset size increases, this bias drives generation toward overly smooth, homogeneous samples, reducing intra-class diversity and limiting generalization. We present PRISM (PRIors from diverse Source Models), a framework that disentangles architectural priors during synthesis. PRISM decouples the logit-matching and regularization objectives, supervising them with different teacher architectures: a primary model for logits and a stochastic subset for batch-normalization (BN) alignment. On ImageNet-1K, PRISM consistently and reproducibly outperforms single-teacher methods (e.g., SRe2L) and recent multi-teacher variants (e.g., G-VBSM) at low- and mid-IPC regimes. The generated data also show significantly richer intra-class diversity, as reflected by a notable drop in cosine similarity between features. We further analyze teacher selection strategies (pre- vs. intra-distillation) and introduce a scalable cross-class batch formation scheme for fast parallel synthesis. Code will be released after the review period.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。