让联邦学习模型按需快慢推理,提升效率与精度。
Federated Personalization of Early-Exit Networks
- 用分层协同机制解决不同客户端间的模型冲突。
- 在多个数据集上实现30.79%~46.86%的推理成本降低。
- 适合资源受限场景下个性化模型部署的开发者。
个性化联邦学习(PFL)能针对客户端数据特点定制模型,对去中心化异构环境至关重要,但现有方法生成静态模型,难以适应不同上下文和资源条件下的推理需求。早期退出网络(EENs)通过中间分类器实现自适应推理,具有潜力。然而将EEN引入PFL带来双重挑战:客户端间差异与各层级退出目标的冲突。此前研究无法同时解决这两类问题,导致性能下降。本文提出X-FED框架,通过深度优先的学生协调机制缓解浅层与深层退出间的干扰,并实现跨客户端有效知识迁移。此外,采用客户端解耦设计,理论支持下降低通信开销。大量实验表明,相比先进PFL及PFL-EE方法,X-FED在保持更高准确率的同时,推理成本降低30.79%至46.86%。
原文摘要 · Abstract (English)
Personalized Federated Learning (PFL) excels at tailoring client-specific models, which is particularly critical for decentralized and heterogeneous data environments, yet existing methods produce static models with a fixed tradeoff between accuracy and efficiency. This inherent static nature limits their ability to adapt to inference demands that vary with context and resource availability, posing a challenge for real-world deployment. Early-exit networks (EENs), which enable adaptive inference via intermediate classifiers, offer a promising solution. However, integrating EENs into PFL introduces two intertwined conflicts: client-wise heterogeneity across clients and depth-wise interference arising from conflicting exit objectives. Prior studies fail to resolve both conflicts simultaneously, leading to suboptimal performance. In this paper, we propose X-FED, a novel Conflict-Aware Cross-Client Federated Exit Distillation framework that jointly addresses both client- and depth-wise conflicts while extending PFL to early-exit networks. At its core, X-FED employs a progressive, depth-prioritized student coordination mechanism that mitigates interference among shallow and deep exits while enabling effective personalized knowledge transfer across clients. Furthermore, we introduce a client-decoupled formulation that reduces communication overhead with theoretical soundness. Extensive evaluations on various datasets show that compared to state-of-the-art PFL and PFL-EE methods, X-FED achieves higher accuracy while reducing inference costs by 30.79%-46.86%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。