提出对抗蒸馏采样，让模型输出难被复制却仍好用。
Antidistillation Sampling
- 通过修改下一词概率分布，干扰推理轨迹
- 使蒸馏效果显著下降但模型性能不变
- 适合担心模型被复制的开发者使用
前沿模型生成长推理轨迹时，会产出丰富的词元序列，容易被用于模型蒸馏。意识到这一漏洞，模型所有者可能希望采用采样策略，在不损害模型实际性能的前提下，降低蒸馏有效性。抗蒸馏采样（Antidistillation Sampling）正是为此而设计：通过战略性地修改模型的下一词概率分布，污染推理轨迹，使其在蒸馏任务中表现显著下降，同时保持模型的实际可用性。更多详情请见 https://antidistillation.com。
原文摘要 · Abstract (English)
Frontier models that generate extended reasoning traces inadvertently produce rich token sequences that can facilitate model distillation. Recognizing this vulnerability, model owners may seek sampling strategies that limit the effectiveness of distillation without compromising model performance. Antidistillation sampling provides exactly this capability. By strategically modifying a model's next-token probability distribution, antidistillation sampling poisons reasoning traces, rendering them significantly less effective for distillation while preserving the model's practical utility. For further details, see https://antidistillation.com.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。