arXiv:2409.10570cs.LGcs.AI2024-09被引 2

无需训练即可为医疗大模型嵌入版权水印,防窃取且高效。

Protecting Copyright of Medical Pre-trained Language Models: Training-Free Backdoor Model Watermarking

  • 用低频词作触发器,替换词向量嵌入水印。
  • 水印在医疗任务中表现稳定,10秒完成嵌入。
  • 抗模型提取、剪枝等攻击,适合医疗领域应用。

随着智能医疗的发展,医学预训练语言模型(Med-PLMs)在下游医疗任务中展现出显著效果。然而,这些模型易被滥用和盗用,亟需版权保护。现有预训练语言模型水印方法因领域-任务不匹配及水印嵌入效率低,难以直接应用于Med-PLMs。为此,我们提出首个无需训练的医用水印方法。该方法采用低频词作为触发器,通过替换其在模型词嵌入层中的向量为特定医学术语向量来嵌入水印。水印后的Med-PLMs对触发器的输出与对应医学术语一致。基于此独特映射关系,我们设计了适配不同下游任务的水印提取方案,解决了以往方法的领域-任务不匹配问题。实验表明,该方法在各类医疗下游任务中均具优越性。同时,该方法对模型提取、剪枝及融合式后门移除攻击具有鲁棒性,且嵌入效率极高,仅需10秒。

原文摘要 · Abstract (English)

With the advancement of intelligent healthcare, medical pre-trained language models (Med-PLMs) have emerged and demonstrated significant effectiveness in downstream medical tasks. While these models are valuable assets, they are vulnerable to misuse and theft, requiring copyright protection. However, existing watermarking methods for pre-trained language models (PLMs) cannot be directly applied to Med-PLMs due to domain-task mismatch and inefficient watermark embedding. To fill this gap, we propose the first training-free backdoor model watermarking for Med-PLMs. Our method employs low-frequency words as triggers, embedding the watermark by replacing their embeddings in the model's word embedding layer with those of specific medical terms. The watermarked Med-PLMs produce the same output for triggers as for the corresponding specified medical terms. We leverage this unique mapping to design tailored watermark extraction schemes for different downstream tasks, thereby addressing the challenge of domain-task mismatch in previous methods. Experiments demonstrate superior effectiveness of our watermarking method across medical downstream tasks. Moreover, the method exhibits robustness against model extraction, pruning, fusion-based backdoor removal attacks, while maintaining high efficiency with 10-second watermark embedding.

水印技术医疗AI版权保护

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。