首个针对多模态RAG系统的黑盒成员推理攻击方法,可泄露敏感数据隐私。
MrM: Black-Box Membership Inference Attacks against Multimodal RAG Systems
- 通过对象感知扰动与反事实掩码选择,精准诱导系统暴露数据归属信息。
- 在8个主流视觉语言模型上实现样本级与集合级攻击,成功率超90%。
- 适用于评估多模态系统隐私安全,尤其适合关注数据泄露风险的研究者。
多模态检索增强生成(RAG)系统通过融合跨模态知识提升大视觉语言模型性能,广泛应用于真实场景。然而,其知识库可能包含敏感信息,且系统对外部用户开放间接访问,易受成员推理攻击(MIA)威胁。现有方法多聚焦文本模态,视觉模态研究不足。为此,本文提出首个针对多模态RAG系统的黑盒MIA框架MrM,采用多目标数据扰动策略,在反事实攻击约束下,同时触发目标数据召回与成员信息泄露。首先使用对象感知扰动确保关键语义保留并成功检索;其次设计反事实启发的掩码选择策略,优先处理最具信息量的遮蔽区域,减少模型自知干扰;最后通过建模查询行为,从响应模式中提取重建被遮掩语义的特征,进行统计成员推理。在两个视觉数据集和八个主流商业视觉语言模型(如GPT-4o、Gemini-2)上的实验表明,MrM在样本级与集合级评估中均表现强劲,且对自适应防御保持鲁棒性。
原文摘要 · Abstract (English)
Multimodal retrieval-augmented generation (RAG) systems enhance large vision-language models by integrating cross-modal knowledge, enabling their increasing adoption across real-world multimodal tasks. These knowledge databases may contain sensitive information that requires privacy protection. However, multimodal RAG systems inherently grant external users indirect access to such data, making them potentially vulnerable to privacy attacks, particularly membership inference attacks (MIAs). % Existing MIA methods targeting RAG systems predominantly focus on the textual modality, while the visual modality remains relatively underexplored. To bridge this gap, we propose MrM, the first black-box MIA framework targeted at multimodal RAG systems. It utilizes a multi-object data perturbation framework constrained by counterfactual attacks, which can concurrently induce the RAG systems to retrieve the target data and generate information that leaks the membership information. Our method first employs an object-aware data perturbation method to constrain the perturbation to key semantics and ensure successful retrieval. Building on this, we design a counterfact-informed mask selection strategy to prioritize the most informative masked regions, aiming to eliminate the interference of model self-knowledge and amplify attack efficacy. Finally, we perform statistical membership inference by modeling query trials to extract features that reflect the reconstruction of masked semantics from response patterns. Experiments on two visual datasets and eight mainstream commercial visual-language models (e.g., GPT-4o, Gemini-2) demonstrate that MrM achieves consistently strong performance across both sample-level and set-level evaluations, and remains robust under adaptive defenses.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。