根据客户端数据差异智能分配模型大小,提升边缘联邦学习效果
HASA: Subnet Allocation for Compute-Constrained Model-Heterogeneous Federated Learning

- 基于本地数据异构性动态分配子网宽度,兼顾算力约束
- 在7个客户端任务中平均准确率提升0.5个百分点,最差客户端表现更优
- 适合资源不均且数据分布差异大的边缘设备联邦学习场景
边缘服务越来越多地采用联邦学习,在本地个性化模型的同时保护敏感数据。实际部署需应对客户端资源与本地数据分布的双重异构性。模型异构联邦学习通过允许每个客户端训练共享超网的子网来降低客户端开销,但多数子网分配策略仅考虑设备约束,未显式处理统计异构性。本文提出异构感知子网分配(HASA),一种仅在训练阶段使用的规则:根据本地训练数据计算的异构性得分分配子网宽度,同时满足固定加权算力预算。该设计支持与其它分配策略的预算匹配比较。在包含7个客户端的文章标题下一个词预测基准上,HASA相比均匀分配在10组匹配种子下将平均客户端测试准确率从13.82%提升至14.32%,并改善了最差客户端的准确率。与代表性部分训练基线在匹配预算下对比,HASA在该基准上取得了最强的最差客户端和尾部客户端准确率。方向性消融实验表明,将更小子网分配给异构性更高的客户端会降低平均性能和尾部性能。跨领域图像分类研究进一步表明,异构性感知分配的有效性取决于异构性评分是否准确反映客户端对额外模型宽度的需求。
原文摘要 · Abstract (English)
Edge services increasingly use federated learning to personalize on-device models while keeping sensitive data local. In practice, deployments must handle heterogeneity in both client resources and local data distributions. Model-heterogeneous federated learning lowers client cost by allowing each client to train a subnet of a shared supernet, but most subnet-allocation policies are driven by device constraints and do not explicitly account for statistical heterogeneity. This paper proposes Heterogeneity-Aware Subnet Allocation (HASA), a train-only rule that assigns subnet widths based on client heterogeneity scores computed from local training data while enforcing a fixed size-weighted compute budget. This design enables budget-matched comparisons with alternative allocation policies. On an article-title next-word prediction benchmark with seven clients, HASA improves unweighted mean client test accuracy over uniform allocation across 10 matched seeds, increasing mean client test accuracy from 13.82 percent to 14.32 percent, and improves worst-client accuracy on average. In a matched-budget comparison with representative partial-training baselines, HASA achieves the strongest worst-client and tail-client accuracy on this benchmark. A directionality ablation shows that assigning smaller subnets to more heterogeneous clients degrades both mean and tail performance. A cross-domain image-classification study further shows that the effectiveness of heterogeneity-aware allocation depends on how well the heterogeneity score reflects clients' need for additional model width.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。