arXiv:2607.15467cs.LGcs.CR2026-07

提出ADS-C防御方法,零成本阻止分类模型被窃取。

ADS-C: Antidistillation Sampling for Classification

论文配图:ADS-C: Antidistillation Sampling for Classification
图 1 · 摘自论文原文
  • 基于输入置信度差构建动态扰动,保护分类器不被模仿
  • 在CIFAR-100等数据集上使学生模型准确率下降17.4~29.6个百分点
  • 防御不损失教师模型精度,且让攻击者失去窃取动机

知识蒸馏使攻击者可通过查询预测接口并训练代理模型来复制专有分类器。本文将大语言模型中的反蒸馏采样机制迁移至分类任务,发现其行为由教师模型的每输入置信度差分布决定。由于训练良好的分类器极度自信,直接迁移存在惰性窗口:低于闭式可预测阈值时,对攻防双方均无影响;超过后防御发生相变,导致教师性能退化快于学生。温度软化可闭式重标转移临界点,所有配置均处于不利权衡曲线。我们提出ADS-C,在每输入置信度差预算下构造扰动,严格保证所有服务标签不变,因此防御后教师准确率与原始一致。在此保障下,蒸馏出的学生在CIFAR-100、CIFAR-10、Tiny-ImageNet上分别下降17.4、29.6、13.3个百分点;若用原方法达到同等破坏需牺牲教师27.5、32.9、22.2个百分点准确率。因服务标签未变,硬标签攻击无效;而防御后的软输出使学生最高低29.7个百分点——窃取激励不仅消除,反而逆转。据我们所知,ADS-C是首个分类任务中实用代价为零的反蒸馏防御。

原文摘要 · Abstract (English)

Knowledge distillation enables an adversary to replicate a proprietary classifier by querying its prediction interface and training a surrogate on the returned probability vectors. Antidistillation sampling, proposed for large language models, counters this threat with an input-dependent, gradient-directed perturbation of the served distribution; its transfer to classification has not been studied. Adapting the defense to classification, we show its behavior is governed by the distribution of the teacher's per-input confidence margins. Because well-trained classifiers are severely overconfident, the direct transfer exhibits an inert window: below a closed-form-predictable threshold, it affects neither attacker nor defender; beyond it, the defense undergoes a phase transition and degrades the teacher faster than the attacker's student. Temperature softening rescales the transition in closed form, and every temperature configuration lies on the same unfavorable trade-off curve. Our method, ADS-C, composes the perturbation under a closed-form, per-input margin budget that provably preserves every served top-1 prediction, so the defended teacher's accuracy equals the undefended teacher's identically. Under this guarantee the distilled student still loses 17.4 percentage points on CIFAR-100, 29.6 on CIFAR-10, and 13.3 on Tiny-ImageNet; matching this degradation with the unmodified defense costs 27.5, 32.9, and 22.2 points of teacher accuracy. Because served labels are unchanged, a hard-label attacker gains nothing, while the defended soft output trains a student up to 29.7 points below that floor: the incentive to distill served probabilities is not merely removed but reversed. To our knowledge, ADS-C is the first antidistillation defense for classification whose utility cost is exactly zero.

模型安全反蒸馏分类防御

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。