提出新联邦学习框架,解决边缘智能中的异构与异步问题。
Invariant Federated Learning for Edge Intelligence: Mitigating Heterogeneity and Asynchrony via Exit Strategy and Invariant Penalty
- 通过退出策略和不变性惩罚缓解客户端异构与异步影响。
- 在不增加通信开销下降低分布外预测误差,提升泛化性能。
- 适合资源受限的边缘智能场景,尤其关注模型鲁棒性与因果性。
本文提出一种面向资源受限边缘智能的不变联邦学习系统,通过退出策略与不变性惩罚缓解异构性与异步性影响。引入参数正交性衡量异构及异步客户端的贡献。理论证明异常客户端退出可保障多数客户端的模型效果。为确保退出异常客户端及资源不足客户端的性能,提出联邦学习不变性惩罚通用化(FedIPG),通过构建不变参数与异构参数的近似正交性实现。理论分析表明,FedIPG在不增加通信负担的前提下降低分布外预测损失。在四个数据集上多尺度实证测试显示,结合退出策略的系统显著提升分布内性能,并优于当前最优算法的分布外泛化能力,同时保持模型收敛性。视觉实验进一步验证了FedIPG对混杂特征的忽略具备初步因果性。
原文摘要 · Abstract (English)
This paper provides an invariant federated learning system for resource-constrained edge intelligence. This framework can mitigate the impact of heterogeneity and asynchrony via exit strategy and invariant penalty. We introduce parameter orthogonality into edge intelligence to measure the contribution or impact of heterogeneous and asynchronous clients. It is proved in this paper that the exit of abnormal edge clients can guarantee the effect of the model on most clients. Meanwhile, to ensure the models' performance on exited abnormal clients and those who lack training resources, we propose Federated Learning with Invariant Penalty for Generalization (FedIPG) by constructing the approximate orthogonality of the invariant parameters and the heterogeneous parameters. Theoretical proof shows that FedIPG reduces the Out-Of-Distribution prediction loss without increasing the communication burden. The performance of FedIPG combined with an exit strategy is tested empirically in multiple scales using four datasets. It shows our system can enhance In-Distribution performance and outperform the state-of-the-art algorithm in Out-Of-Distribution generalization while maintaining model convergence. Additionally, the results of the visual experiment prove that FedIPG contains preliminary causality in terms of ignoring confounding features.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。