提出新方法破解联邦大模型隐私漏洞，无需额外训练即可高效识别数据归属。
Toward Efficient Membership Inference Attacks against Federated Large Language Models: A Projection Residual Approach
- 利用嵌入向量在梯度子空间的投影残差检测数据成员身份
- 在4个基准上实现接近100%攻击准确率，较前人提升75.75%
- 不依赖影子模型或历史更新，适合实际部署场景中的隐私评估
联邦大语言模型（FedLLMs）允许多方在不共享原始数据的前提下协同微调大模型，缓解资源与隐私挑战。尽管数据本地化，共享梯度仍可能通过成员推理攻击（MIAs）泄露敏感信息。然而，由于参数规模大、收敛快及梯度稀疏非正交等特性，现有攻击方法失效。为此，本文提出首个面向FedLLMs的投影残差被动攻击方法ProjRes。该方法以隐层嵌入向量作为样本表示，分析其在梯度子空间的投影残差，揭示梯度与输入间的内在关联。ProjRes无需影子模型、辅助分类器或历史更新，兼具高效性与鲁棒性。在四个基准和四种大模型上的实验表明，其攻击准确率接近100%，相比先前方法最高提升75.75%，且在强差分隐私防御下仍有效。研究揭示了FedLLMs中被忽视的隐私风险，呼吁重新审视其安全假设。代码与数据见链接。
原文摘要 · Abstract (English)
Federated Large Language Models (FedLLMs) enable multiple parties to collaboratively fine-tune LLMs without sharing raw data, addressing challenges of limited resources and privacy concerns. Despite data localization, shared gradients can still expose sensitive information through membership inference attacks (MIAs). However, FedLLMs' unique properties, i.e. massive parameter scales, rapid convergence, and sparse, non-orthogonal gradients, render existing MIAs ineffective. To address this gap, we propose ProjRes, the first projection residuals-based passive MIA tailored for FedLLMs. ProjRes leverages hidden embedding vectors as sample representations and analyzes their projection residuals on the gradient subspace to uncover the intrinsic link between gradients and inputs. It requires no shadow models, auxiliary classifiers, or historical updates, ensuring efficiency and robustness. Experiments on four benchmarks and four LLMs show that ProjRes achieves near 100% accuracy, outperforming prior methods by up to 75.75%, and remains effective even under strong differential privacy defenses. Our findings reveal a previously overlooked privacy vulnerability in FedLLMs and call for a re-examination of their security assumptions. Our code and data are available at $\href{https://anonymous.4open.science/r/Passive-MIA-5268}{link}$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。