arXiv:2602.03812cs.LGcs.AI2026-02被引 4

用更聪明的水印技术,让大模型抄袭更容易被发现,且不影响模型表现。

Antidistillation Fingerprinting

  • 基于梯度优化选择关键标记,主动增强水印可检测性。
  • 在数学、对话、代码生成任务中,水印强度提升但性能损失极小。
  • 适用于未知架构的学生模型,适合模型安全检测场景。

模型蒸馏使得高效模拟前沿大语言模型成为可能,因而亟需可靠的机制来检测第三方学生模型是否基于教师模型输出进行训练。然而,现有指纹技术依赖启发式扰动,导致生成质量与指纹强度之间存在显著权衡,常需大幅降低模型实用性以确保指纹被有效内化。本文提出反蒸馏指纹(ADFP),一种原理上对齐学生模型学习动态的方法。基于反蒸馏采样框架,ADFP利用代理模型识别并采样能最大化学生模型微调后指纹可检测性的标记,而非依赖朴素水印的非目标偏差偶然吸收。在GSM8K、OASST1和MBPP数据集上的实验表明,ADFP相比最先进基线实现显著的帕累托改进,在数学推理、对话和代码生成任务中,以极小代价大幅提升检测置信度,即使学生模型架构未知也适用。

原文摘要 · Abstract (English)

Model distillation enables efficient emulation of frontier large language models (LLMs), creating a need for robust mechanisms to detect when a third-party student model has trained on a teacher model's outputs. However, existing fingerprinting techniques that could be used to detect such distillation rely on heuristic perturbations that impose a steep trade-off between generation quality and fingerprinting strength, often requiring significant degradation of utility to ensure the fingerprint is effectively internalized by the student. We introduce antidistillation fingerprinting (ADFP), a principled approach that aligns the fingerprinting objective with the student's learning dynamics. Building upon the gradient-based framework of antidistillation sampling, ADFP utilizes a proxy model to identify and sample tokens that directly maximize the expected detectability of the fingerprint in the student after fine-tuning, rather than relying on the incidental absorption of the un-targeted biases of a more naive watermark. Experiments on GSM8K, OASST1, and MBPP demonstrate that ADFP achieves a significant Pareto improvement over state-of-the-art baselines, yielding stronger detection confidence with minimal impact on utility across mathematical reasoning, dialogue, and code generation, even when the student model's architecture is unknown.

模型安全指纹检测蒸馏防御

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。