用线性探测分析大模型内部状态，提升成员推理攻击检测能力。
LUMIA: Linear probing for Unimodal and MultiModal Membership Inference Attacks leveraging internal LLM states
- 通过逐层线性探测分析大模型内部激活值，捕捉成员信息痕迹。
- 在单模态任务中平均提升AUC 15.71%，超65%场景达到AUC>60%。
- 首次揭示视觉输入对多模态成员推理的显著影响，85.9%实验达高检测率。
大型语言模型（LLMs）在各类应用中日益普及，但成员推理攻击（MIA）风险也随之上升。以往研究多聚焦于黑盒到灰盒模型，忽略了利用大模型内部信息的潜力。为此，本文提出使用线性探测（LPs）方法，通过分析大模型内部激活值来检测成员推理攻击，命名为LUMIA。该方法对不同模型架构、规模和数据集（包括单模态与多模态任务）进行测试。在单模态任务中，LUMIA相比现有技术平均提升15.71%的曲线下面积（AUC），且在65.33%的案例中实现AUC>60%，较当前最优水平提升46.80%。此外，方法揭示了成员推理最易被检测的模型层级。在多模态模型中，线性探测显示视觉输入显著增强成员推理检测能力——85.90%的实验中达到AUC>60%。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly used in a variety of applications, but concerns around membership inference have grown in parallel. Previous efforts focus on black-to-grey-box models, thus neglecting the potential benefit from internal LLM information. To address this, we propose the use of Linear Probes (LPs) as a method to detect Membership Inference Attacks (MIAs) by examining internal activations of LLMs. Our approach, dubbed LUMIA, applies LPs layer-by-layer to get fine-grained data on the model inner workings. We test this method across several model architectures, sizes and datasets, including unimodal and multimodal tasks. In unimodal MIA, LUMIA achieves an average gain of 15.71 % in Area Under the Curve (AUC) over previous techniques. Remarkably, LUMIA reaches AUC>60% in 65.33% of cases -- an increment of 46.80% against the state of the art. Furthermore, our approach reveals key insights, such as the model layers where MIAs are most detectable. In multimodal models, LPs indicate that visual inputs can significantly contribute to detect MIAs -- AUC>60% is reached in 85.90% of experiments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。