让大模型在联邦学习中安全暴露敏感信息,不降性能还防泄露。
SecureGate: Learning When to Reveal PII Safely via Token-Gated Dual-Adapters for Federated LLMs
- 用双适配器架构区分隐私与非隐私信息,按需控制披露。
- 实测可降低31.66倍的隐私攻击成功率,提取召回率降17.07倍。
- 适合医疗、金融等需严控数据泄露的联邦大模型场景。
联邦学习(FL)可在不共享原始数据的前提下实现跨机构协作训练,适用于高隐私敏感场景。随着大语言模型(LLM)的普及,基于分布式数据的联邦微调受到关注,但面临两大挑战:(i) LLM记忆导致的个人身份信息(PII)泄露,(ii) 异构数据下全局泛化与本地效用间的持续矛盾。现有防护手段如数据清洗和差分隐私虽能减少泄露,但常损害下游性能。本文提出SecureGate,一种面向联邦大模型的隐私感知微调框架,实现细粒度隐私控制而不损失效用。该框架采用双适配器LoRA结构:安全适配器学习可全局共享的净化表示,揭示适配器保留组织特异的敏感知识。一个令牌控制的门控模块在推理时选择性激活适配器,实现无重训练的信息可控披露。多模型、多真实数据集上的实验表明,SecureGate在显著降低PII泄露的同时提升任务效用,使未经授权请求的推理攻击准确率下降31.66倍,信息提取召回率降低17.07倍。同时保持100%适配器路由可靠性,计算与通信开销极低。
原文摘要 · Abstract (English)
Federated learning (FL) enables collaborative training across organizational silos without sharing raw data, making it attractive for privacy-sensitive applications. With the rapid adoption of large language models (LLMs), federated fine-tuning of generative LLMs has gained attention as a way to leverage distributed data while preserving confidentiality. However, this setting introduces fundamental challenges: (i) privacy leakage of personally identifiable information (PII) due to LLM memorization, and (ii) a persistent tension between global generalization and local utility under heterogeneous data. Existing defenses, such as data sanitization and differential privacy, reduce leakage but often degrade downstream performance. We propose SecureGate, a privacy-aware federated fine-tuning framework for LLMs that provides fine-grained privacy control without sacrificing utility. SecureGate employs a dual-adapter LoRA architecture: a secure adapter that learns sanitized, globally shareable representations, and a revealing adapter that captures sensitive, organization-specific knowledge. A token-controlled gating module selectively activates these adapters at inference time, enabling controlled information disclosure without retraining. Extensive experiments across multiple LLMs and real-world datasets show that SecureGate improves task utility while substantially reducing PII leakage, achieving up to a 31.66X reduction in inference attack accuracy and a 17.07X reduction in extraction recall for unauthorized requests. Additionally, it maintains 100% routing reliability to the correct adapter and incurs only minimal computational and communication overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。