用泰勒展开保护大模型权重,防滥用还防窃取
Taylor Unswift: Secured Weight Release for Large Language Models via Taylor Expansion
- 将模型权重转为泰勒级数参数发布,不直接暴露原始权重
- 生成速度降低4倍以上,延迟显著抑制大规模滥用
- 可防御权重重构攻击,适合关注模型版权的开发者
确保释放的大语言模型(LLMs)安全面临重大困境:现有机制要么损害所有权,要么引发数据隐私问题。为此,我们提出TaylorMLP,以保护已发布LLMs的所有权并防止其被滥用。具体而言,TaylorMLP通过将LLM权重转换为泰勒级数参数来保留所有权。开发者无需发布原始权重,而是发布泰勒级数参数,从而保障模型安全。此外,通过增加泰勒级数项数,可主动降低生成速度,诱导低速令牌生成,有效防止潜在的大规模未经授权使用。在五个数据集和三种LLM架构上的实验表明,TaylorMLP使延迟提升超过4倍,同时生成的令牌与原模型完全一致。后续防御实验进一步验证,该方法能有效阻止用户基于下游数据集重构权重值。
原文摘要 · Abstract (English)
Ensuring the security of released large language models (LLMs) poses a significant dilemma, as existing mechanisms either compromise ownership rights or raise data privacy concerns. To address this dilemma, we introduce TaylorMLP to protect the ownership of released LLMs and prevent their abuse. Specifically, TaylorMLP preserves the ownership of LLMs by transforming the weights of LLMs into parameters of Taylor-series. Instead of releasing the original weights, developers can release the Taylor-series parameters with users, thereby ensuring the security of LLMs. Moreover, TaylorMLP can prevent abuse of LLMs by adjusting the generation speed. It can induce low-speed token generation for the protected LLMs by increasing the terms in the Taylor-series. This intentional delay helps LLM developers prevent potential large-scale unauthorized uses of their models. Empirical experiments across five datasets and three LLM architectures demonstrate that TaylorMLP induces over 4x increase in latency, producing the tokens precisely matched with original LLMs. Subsequent defensive experiments further confirm that TaylorMLP effectively prevents users from reconstructing the weight values based on downstream datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。