攻击者可伪造模型诱骗合并，窃取用户隐私数据。
Be Cautious When Merging Unfamiliar LLMs: A Phishing Model Capable of Stealing Privacy
- 设计钓鱼指令数据集训练恶意模型，伪装成正常模型
- 合并后导致个人身份信息泄露率上升3.9%，成员信息泄露升17.4%
- 适用于关注模型安全与隐私保护的研究者与开发者
模型合并是将多个任务专用大模型整合为统一模型的技术，使合并后的模型继承各模型的专长。然而，多数任务专用模型来自开源社区且未经严格审计，存在安全隐患。本文揭示了一项被忽视的隐私风险：一个不安全模型可能损害参与合并的其他模型的隐私。为此，我们提出PhiMM攻击方法，通过构建隐私钓鱼指令数据集训练出能够窃取隐私的钓鱼模型，并引入新型模型伪装技术，模仿特定能力以隐藏攻击意图，诱导用户合并该模型。一旦合并，攻击者可通过钓鱼指令查询合并模型，提取个人身份信息（PII）或推断成员信息（MI）。实验表明，合并钓鱼模型后，平均PII泄漏率上升3.9%，成员信息泄漏率上升17.4%。相关代码已公开。
原文摘要 · Abstract (English)
Model merging is a widespread technology in large language models (LLMs) that integrates multiple task-specific LLMs into a unified one, enabling the merged model to inherit the specialized capabilities of these LLMs. Most task-specific LLMs are sourced from open-source communities and have not undergone rigorous auditing, potentially imposing risks in model merging. This paper highlights an overlooked privacy risk: \textit{an unsafe model could compromise the privacy of other LLMs involved in the model merging.} Specifically, we propose PhiMM, a privacy attack approach that trains a phishing model capable of stealing privacy using a crafted privacy phishing instruction dataset. Furthermore, we introduce a novel model cloaking method that mimics a specialized capability to conceal attack intent, luring users into merging the phishing model. Once victims merge the phishing model, the attacker can extract personally identifiable information (PII) or infer membership information (MI) by querying the merged model with the phishing instruction. Experimental results show that merging a phishing model increases the risk of privacy breaches. Compared to the results before merging, PII leakage increased by 3.9\% and MI leakage increased by 17.4\% on average. We release the code of PhiMM through a link.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。