发现语言模型编辑会泄露敏感数据,提出可逆向还原的攻击方法
Reverse-Engineering Model Editing on Language Models
- 利用参数更新的低秩结构,通过谱分析还原被编辑主体
- 基于熵减少的提示重构攻击,高成功率恢复编辑语义上下文
- 适合关注模型安全与隐私保护的研究者阅读
大型语言模型在万亿级语料上预训练,不可避免地记忆敏感信息。主流的定位-编辑方法通过修改参数实现模型编辑而无需重新训练,但本文揭示该范式存在关键漏洞:参数更新无意中成为侧信道,使攻击者可恢复编辑内容。我们提出两阶段逆向工程攻击KSTER(KeySpaceReconstruction-then-EntropyReduction),首先理论上证明更新矩阵的行空间编码了被编辑主体的“指纹”,可通过谱分析准确恢复;其次引入基于熵的提示恢复攻击,重建编辑的语义上下文。在多个LLM上的实验表明,该攻击可高成功率恢复编辑数据。此外,我们提出子空间伪装防御策略,通过语义伪影混淆更新指纹,在不损害编辑效用的前提下有效降低重建风险。代码已开源。
原文摘要 · Abstract (English)
Large language models (LLMs) are pretrained on corpora containing trillions of tokens and, therefore, inevitably memorize sensitive information. Locate-then-edit methods, as a mainstream paradigm of model editing, offer a promising solution by modifying model parameters without retraining. However, in this work, we reveal a critical vulnerability of this paradigm: the parameter updates inadvertently serve as a side channel, enabling attackers to recover the edited data. We propose a two-stage reverse-engineering attack named \textit{KSTER} (\textbf{K}ey\textbf{S}paceRecons\textbf{T}ruction-then-\textbf{E}ntropy\textbf{R}eduction) that leverages the low-rank structure of these updates. First, we theoretically show that the row space of the update matrix encodes a ``fingerprint" of the edited subjects, enabling accurate subject recovery via spectral analysis. Second, we introduce an entropy-based prompt recovery attack that reconstructs the semantic context of the edit. Extensive experiments on multiple LLMs demonstrate that our attacks can recover edited data with high success rates. Furthermore, we propose \textit{subspace camouflage}, a defense strategy that obfuscates the update fingerprint with semantic decoys. This approach effectively mitigates reconstruction risks without compromising editing utility. Our code is available at https://github.com/reanatom/EditingAttack.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。