用少量真实样本生成逼真数据，实现无数据知识蒸馏
C2G-KD: PCA-Constrained Generator for Data-Free Knowledge Distillation
- 用主成分分析约束生成数据分布，保持类别结构一致
- 仅需每类2个真实样本即可生成有效合成数据，准确率超90%
- 适合无原始数据可用的模型压缩场景
我们提出C2G-KD，一种无需真实训练数据的知识蒸馏框架。通过冻结的教师模型与从每类仅2个真实样本中估计的主成分分析（PCA）几何约束，训练一个类别条件生成器，使其生成的合成样本能激活教师模型输出。该方法在不接触真实数据的前提下，结合语义与结构损失，使生成样本位于特定类别的PCA子空间内，从而保持拓扑一致性与多样性。在MNIST上的实验表明，即使仅有极少类结构信息，也能构建出有效的合成训练流程。
原文摘要 · Abstract (English)
We introduce C2G-KD, a data-free knowledge distillation framework where a class-conditional generator is trained to produce synthetic samples guided by a frozen teacher model and geometric constraints derived from PCA. The generator never observes real training data but instead learns to activate the teacher's output through a combination of semantic and structural losses. By constraining generated samples to lie within class-specific PCA subspaces estimated from as few as two real examples per class, we preserve topological consistency and diversity. Experiments on MNIST show that even minimal class structure is sufficient to bootstrap useful synthetic training pipelines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。