arXiv:2605.18829cs.LGcs.CR2026-05

提出LADS采样方法,防模型蒸馏攻击且不影响正常用户体验。

Lossless Anti-Distillation Sampling

论文配图:Lossless Anti-Distillation Sampling
图 1 · 摘自论文原文
  • 基于查询语义和请求次数生成私有随机种子,实现无损采样
  • 多账号蒸馏时样本相关性上升,导致学生模型泛化能力下降
  • 适用于需要防御数据盗用的生成式AI服务部署

前沿商业生成模型正面临蒸馏攻击威胁:攻击者通过收集生成结果,以极低成本训练竞争模型。现有防御要么修改输出影响用户体验,要么依赖行为检测易被多账号绕过。本文提出无损反蒸馏采样(LADS),一种新型采样方案,可有效抵御多账号蒸馏,同时对合法用户保持无损体验。LADS将每次生成的随机性由查询语义与用户请求次数决定的私有种子生成。每个合法用户每次访问均独立采样自原模型,无质量损失;而攻击者使用不同账号时,若查询语义相同,则共享隐含随机性,导致采集数据相关,降低样本多样性,削弱学生模型泛化能力。基于均匀收敛理论,证明在无条件与条件生成场景下,LADS显著降低蒸馏模型的泛化误差收敛速度。图像生成、数学推理与代码生成实验表明,LADS显著削弱蒸馏模型性能,同时保证单个用户的统计特性完全一致。

原文摘要 · Abstract (English)

Frontier commercial generative models face a growing threat from distillation, whereby a distiller harvests generated responses and trains a competing model of its own at drastically lower cost. Existing defenses either rely on modifying the models outputs, thereby sacrificing response quality for benign users, or on behavioral detection methods, which can be readily circumvented by distributing queries across multiple accounts. In this work, we propose Lossless Anti-Distillation Sampling (LADS), a novel sampling scheme specifically designed to counter multi-account distillation while maintaining a lossless experience for benign users. Concretely, LADS derives the randomness underlying each generation from a private seed determined by the semantic content of the query and the number of times the user has queried the model. By construction, every benign user receives a response independently sampled from the original model at each visit, and thus experiences no distortion. In contrast, for a distiller, different accounts share latent randomness whenever their queries fall in the same semantic bucket. As a result, the harvested data becomes correlated, potentially reducing sample diversity and degrading generalization. Using uniform convergence theory, we show that LADS provably degrades the convergence rate of the distillers generalization gap relative to standard i.i.d. sampling in both unconditional and conditional generation settings. Experiments on image generation, mathematical reasoning, and code generation confirm that LADS substantially degrades the performance of distilled students while preserving exact statistical fidelity for individual users.

生成模型模型安全反蒸馏采样策略

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。