arXiv:2512.03079cs.CLcs.CR2025-12

为嵌入服务模型设计抗改写水印,防止他人黑盒克隆。

Watermarks for Embeddings-as-a-Service Large Language Models

  • 通过线性变换对嵌入向量加水印,验证时逆变换比对相似度。
  • 现有水印可被改写输入文本轻松移除,暴露严重漏洞。
  • 新方法对改写攻击鲁棒,验证准确率接近完美,适合保护模型版权。

大型语言模型在自然语言理解与生成方面表现出色,基于此,企业已推出嵌入即服务(EaaS)以提供文本嵌入的特征提取能力,助力下游NLP任务。然而,已有研究证明EaaS易受黑盒克隆攻击,攻击者无需访问模型内部即可复制其功能。为此,通过在文本嵌入中添加水印来保护服务提供商的知识产权成为应对策略。本论文聚焦于防御此类克隆攻击,揭示了现有EaaS水印的新漏洞:攻击者可通过改写输入文本绕过当前最先进的水印技术。实验表明,在不同改写方式、模型及数据集下,该方法在多数情况下均有效。为此,我们提出新型水印技术WET(Watermarking EaaS with Linear Transformation),采用嵌入向量的线性变换,验证时通过反变换恢复并比较相似度。结果表明,WET对改写攻击具有高度鲁棒性,验证准确率近乎完美。我们还进行了详尽的消融实验,评估各组件与超参数的影响。

原文摘要 · Abstract (English)

Large Language Models (LLMs) have demonstrated exceptional capabilities in natural language understanding and generation. Based on these LLMs, businesses have started to provide Embeddings-as-a-Service (EaaS), offering feature extraction capabilities (in the form of text embeddings) that benefit downstream natural language processing tasks. However, prior research has demonstrated that EaaS is vulnerable to imitation attacks, where an attacker clones the service's model in a black-box manner without access to the model's internal workings. In response, watermarks have been added to the text embeddings to protect the intellectual property of EaaS providers by allowing them to check for model ownership. This thesis focuses on defending against imitation attacks by investigating EaaS watermarks. To achieve this goal, we unveil novel attacks and propose and validate new watermarking techniques. Firstly, we show that existing EaaS watermarks can be removed through paraphrasing the input text when attackers clone the model during imitation attacks. Our study illustrates that paraphrasing can effectively bypass current state-of-the-art EaaS watermarks across various attack setups (including different paraphrasing techniques and models) and datasets in most instances. This demonstrates a new vulnerability in recent EaaS watermarking techniques. Subsequently, as a countermeasure, we propose a novel watermarking technique, WET (Watermarking EaaS with Linear Transformation), which employs linear transformation of the embeddings. Watermark verification is conducted by applying a reverse transformation and comparing the similarity between recovered and original embeddings. We demonstrate its robustness against paraphrasing attacks with near-perfect verifiability. We conduct detailed ablation studies to assess the significance of each component and hyperparameter in WET.

水印嵌入服务模型安全防克隆

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。