通过分片梯度实现无服务器联邦学习的无限扩展。
Shard the Gradient, Scale the Model: Serverless Federated Aggregation via Gradient Partitioning
- 将梯度切分为M份，由独立函数分别平均，避免单个函数内存超限。
- 在500MB以上梯度时成本降低2.7倍，5GB模型仍可部署。
- 适合需要超大规模模型训练的无服务器联邦学习场景。
无服务器平台上的联邦学习聚合面临难以突破的可扩展性瓶颈：现有架构（如lambda-FL、LIFL）将客户端分配给多个聚合器，但每个聚合器必须在内存中保存完整的模型梯度。当梯度超过单函数内存限制（如AWS Lambda的10 GB）时，无论树深度或分支因子如何，聚合均不可行。我们提出GradsSharding，将梯度张量分割为M个分片，每个分片由一个无服务器函数独立平均，接收所有客户端贡献。由于FedAvg是逐元素平均，结果与树形结构完全一致，模型精度不变。单函数内存需求为O(|θ|/M)，与客户端数量无关，支持任意大模型的聚合。我们在HPC实验和真实AWS Lambda部署中评估了GradsSharding，覆盖43 MB至5 GB模型规模。结果表明，梯度大小约500 MB时出现成本拐点，VGG-16规模下成本降低2.7倍，且只有GradsSharding能在无服务器内存上限之外持续部署。
原文摘要 · Abstract (English)
Federated learning (FL) aggregation on serverless platforms faces a hard scalability ceiling: existing architectures (lambda-FL, LIFL) partition clients across aggregators, but every aggregator must hold the complete model gradient in memory. When gradients exceed the per-function memory limit (e.g., 10 GB on AWS Lambda), aggregation becomes infeasible regardless of tree depth or branching factor. We propose GradsSharding, which instead partitions the gradient tensor into M shards, each averaged independently by a serverless function that receives contributions from all clients. Because FedAvg averaging is element-wise, this produces bit-identical results to tree-based approaches, so model accuracy is invariant by construction. Per-function memory is bounded at O(|θ|/M), independent of client count, enabling aggregation of arbitrarily large models. We evaluate GradsSharding against lambda-FL and LIFL through HPC experiments and real AWS Lambda deployments across model sizes from 43 MB to 5 GB. Results show a cost crossover at approximately 500 MB gradient size, 2.7x cost reduction at VGG-16 scale, and that GradsSharding is the only architecture that remains deployable beyond the serverless memory ceiling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。