揭示多模态大模型隐私漏洞及应对挑战
Unveiling Privacy Risks in Multi-modal Large Language Models: Task-specific Vulnerabilities and Mitigation Challenges
- 构建MM-Privacy数据集，评估多模态任务中的泄露与留存风险
- 实测多款MMLLM在图像中敏感信息泄露，任务不一致加剧风险
- 适合关注模型隐私安全的研究者与应用开发者
文本类大语言模型的隐私风险已有深入研究，尤其其记忆并泄露敏感信息的倾向。然而，处理文本与图像的多模态大语言模型（MLLMs）引入了独特的隐私挑战，仍待深入探索。相比纯文本模型，MLLMs可从图像中提取并暴露嵌入的敏感信息，带来新风险。本文揭示部分MLLMs存在隐私泄露问题，会泄漏图像中的敏感数据或内存中存储的信息。具体贡献包括：(1) 提出MM-Privacy，一个用于评估多种多模态任务与场景下隐私风险的综合性数据集，定义了披露风险与留存风险；(2) 基于该数据集系统评估不同MLLMs，展示模型在各类任务中泄露敏感数据的现象；(3) 揭示任务不一致性在隐私风险中的作用，强调亟需制定缓解策略。研究结果凸显了MLLMs的隐私隐患，凸显了防范数据暴露的必要性。数据集与代码已公开。
原文摘要 · Abstract (English)
Privacy risks in text-only Large Language Models (LLMs) are well studied, particularly their tendency to memorize and leak sensitive information. However, Multi-modal Large Language Models (MLLMs), which process both text and images, introduce unique privacy challenges that remain underexplored. Compared to text-only models, MLLMs can extract and expose sensitive information embedded in images, posing new privacy risks. We reveal that some MLLMs are susceptible to privacy breaches, leaking sensitive data embedded in images or stored in memory. Specifically, in this paper, we (1) introduce MM-Privacy, a comprehensive dataset designed to assess privacy risks across various multi-modal tasks and scenarios, where we define Disclosure Risks and Retention Risks. (2) systematically evaluate different MLLMs using MM-Privacy and demonstrate how models leak sensitive data across various tasks, and (3) provide additional insights into the role of task inconsistency in privacy risks, emphasizing the urgent need for mitigation strategies. Our findings highlight privacy concerns in MLLMs, underscoring the necessity of safeguards to prevent data exposure. Our dataset and code can be found here.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。