arXiv:2602.15143cs.AIcs.CL2026-02ACL被引 6

通过重写推理过程,防止大模型知识被非法复制。

Protecting Language Models Against Unauthorized Distillation through Trace Rewriting

  • 用指令动态改写教师模型的推理路径,保持答案正确性。
  • 可使学生模型训练效果下降70%以上,且水印检测误报率近乎为零。
  • 适合关注模型版权保护的研究者与开发者使用。

知识蒸馏广泛用于将大语言模型的能力迁移到更小、更高效的学生模型。然而,未经授权的知识蒸馏不公平地利用了前沿模型开发所投入的巨大努力和成本。本文研究通过修改教师模型生成的推理轨迹,实现两个目标:(1) 抗蒸馏,即降低查询响应对训练的有用性;(2) API水印,即在学生模型中嵌入可验证的签名。我们提出多种动态重写教师推理输出的方法,同时保持答案正确性和语义连贯性。其中两种方法利用大模型自身的重写能力,其余采用基于梯度的技术。实验表明,一种简单的指令式重写方法在显著削弱蒸馏效果的同时,甚至能提升教师性能。此外,该方法可可靠嵌入水印,检测几乎无误报。代码已公开于 https://github.com/xhOwenMa/trace-rewriting。

原文摘要 · Abstract (English)

Knowledge distillation is a widely adopted technique for transferring capabilities from LLMs to smaller, more efficient student models. However, unauthorized use of knowledge distillation takes unfair advantage of the considerable effort and cost put into developing frontier models. We investigate methods for modifying teacher-generated reasoning traces to achieve two objectives that deter unauthorized distillation: (1) \emph{anti-distillation}, or degrading the training usefulness of query responses, and (2) \emph{API watermarking}, which embeds verifiable signatures in student models. We introduce several approaches for dynamically rewriting a teacher's reasoning outputs while preserving answer correctness and semantic coherence. Two of these leverage the rewriting capabilities of LLMs, while others use gradient-based techniques. Our experiments show that a simple instruction-based rewriting approach achieves a strong anti-distillation effect while maintaining or even improving teacher performance. Furthermore, we show that our rewriting approach also enables embedding watermarks that can be reliably detected with essentially no false alarms. Our code is available at https://github.com/xhOwenMa/trace-rewriting.

模型保护知识蒸馏水印技术

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。