让多方在不暴露数据和模型的前提下，安全地共同优化大模型。
A Framework for Double-Blind Federated Adaptation of Foundation Models
- 用多项式近似和低秩适配器改造模型，支持全同态加密计算。
- 分两阶段训练适配器，无需反向传播大模型，降低风险。
- 通过数据打乱和随机块采样防模型窃取，适合隐私敏感场景。
基础模型（FMs）在零样本任务中表现优异，但需针对具体任务微调。然而，隐私限制使得多方数据无法共享，而专有协议又禁止学习服务提供商（LSP）共享模型。本文提出BlindFed框架，实现多方协作的模型适应，同时保护双方隐私：数据所有者既不接触模型也不访问彼此数据，而LSP也无法获取敏感任务数据。BlindFed基于全同态加密（FHE），包含三项创新：(i) 通过多项式逼近与低秩适配器实现对FHE友好的结构改造；(ii) 采用两阶段分裂学习，结合离线知识蒸馏与在线加密推理，实现适配器训练而无需对基础模型进行反向传播；(iii) 引入样本打乱与随机块采样策略，增强抗模型提取攻击能力。在四个图像分类数据集上的实验验证了BlindFed的可行性，尽管存在较高的通信开销和较大的计算复杂度。
原文摘要 · Abstract (English)
Foundation models (FMs) excel in zero-shot tasks but benefit from task-specific adaptation. However, privacy concerns prevent data sharing among multiple data owners, and proprietary restrictions prevent the learning service provider (LSP) from sharing the FM. In this work, we propose BlindFed, a framework enabling collaborative FM adaptation while protecting both parties: data owners do not access the FM or each other's data, and the LSP does not see sensitive task data. BlindFed relies on fully homomorphic encryption (FHE) and consists of three key innovations: (i) FHE-friendly architectural modifications via polynomial approximations and low-rank adapters, (ii) a two-stage split learning approach combining offline knowledge distillation and online encrypted inference for adapter training without backpropagation through the FM, and (iii) a privacy-boosting scheme using sample permutations and stochastic block sampling to mitigate model extraction attacks. Empirical results on four image classification datasets demonstrate the practical feasibility of the BlindFed framework, albeit at a high communication cost and large computational complexity for the LSP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。