通过分片聚合提升联邦学习隐私与可扩展性
ERIS: Enhancing Privacy and Scalability in Federated Learning via Federated Shard Aggregation
- 将客户端更新分片,由多个本地聚合器并行处理
- 在保持模型精度的同时降低通信开销和信息泄露
- 适合大规模联邦学习,尤其关注隐私保护的场景
将联邦学习(FL)扩展到十亿参数模型时,面临隐私、可扩展性与模型效用之间的权衡。现有方法常孤立解决其中问题,牺牲精度、依赖昂贵加密工具,或引入通信与优化效率低下问题。我们提出ERIS框架,核心是联邦分片聚合(FSA)机制:将每个客户端更新划分为非重叠分片,由多个客户端侧聚合器分布式聚合。该机制消除中心聚合瓶颈,限制单个观察者可见信息,且重组后保持集中式更新特性。ERIS还可集成分布式偏移压缩(DSC),进一步减少传输数据量和暴露坐标。我们证明,在标准假设下ERIS保持收敛性,并通过可观测更新比例控制互信息泄漏,随客户端聚合器数量增加而下降,启用DSC时也随压缩程度提升而减少。图像与文本任务实验(包括大语言模型)表明,ERIS在达到与FedAvg相当的性能同时,显著缓解通信瓶颈,并增强对成员推断与重构攻击的鲁棒性,无需依赖复杂加密或降低性能的扰动。
原文摘要 · Abstract (English)
Scaling Federated Learning (FL) to billion-parameter models forces a challenging trade-off between privacy, scalability, and model utility. Existing solutions often tackle these challenges in isolation, sacrificing accuracy, relying on costly cryptographic tools, or introducing communication and optimization inefficiencies that affect convergence. We introduce ERIS, an FL framework centered on Federated Shard Aggregation (FSA), a novel mechanism that partitions each client update into non-overlapping shards whose aggregation is distributed across multiple client-side aggregators. FSA removes the central aggregation bottleneck, limits the information visible to any single observer, and preserves the centralized FL update after reassembly. ERIS can further readily integrate Distributed Shifted Compression (DSC) to reduce transmitted payloads and exposed coordinates. We prove that ERIS preserves convergence under standard assumptions and bounds mutual information leakage by the observable fraction of each update, decreasing with the number of client-side aggregators, and with the compression level when DSC is enabled. Experiments across image and text tasks, including large language models, show that ERIS achieves FedAvg-level utility while substantially reducing communication bottlenecks and improving robustness to membership inference and reconstruction attacks, without relying on heavy cryptography or utility-degrading perturbations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。