FedRIR通过重构信息表示,同时提升联邦学习的全局泛化与本地个性化。
FedRIR: Rethinking Information Representation in Federated Learning
- 分离客户端特定特征,增强本地个性化
- 提炼全局共享特征,提升模型泛化能力
- 适合需要兼顾个性化与通用性的边缘设备场景
网络边缘的移动和物联网(WoT)设备生成大量数据用于机器学习,但隐私问题阻碍了集中式训练。联邦学习(FL)使客户端(设备)在不传输私有数据的情况下协同训练共享模型,但客户端间固有的统计异质性带来挑战,常导致个性化本地模型与全局通用模型之间的权衡。现有方法通常偏向于全局泛化或本地个性化,限制了多样化客户端数据的潜力。为此,我们提出新框架FedRIR,通过重构联邦学习中的信息表示,同时提升全局泛化与本地个性化。具体地,引入掩码客户端特定学习(MCSL),提取细粒度客户端特异性特征,增强个性化;同时设计信息蒸馏模块(IDM),过滤冗余客户端特异性信息,获得更纯净、稳健的全局表示,提升泛化性。将精炼的全局特征与隔离的客户端特定特征融合,构建丰富表示,有效捕捉全局模式与局部差异,显著提升下游任务性能。代码已开源:https://github.com/Deep-Imaging-Group/FedRIR。
原文摘要 · Abstract (English)
Mobile and Web-of-Things (WoT) devices at the network edge generate vast amounts of data for machine learning applications, yet privacy concerns hinder centralized model training. Federated Learning (FL) allows clients (devices) to collaboratively train a shared model coordinated by a central server without transfer private data, but inherent statistical heterogeneity among clients presents challenges, often leading to a dilemma between clients' needs for personalized local models and the server's goal of building a generalized global model. Existing FL methods typically prioritize either global generalization or local personalization, resulting in a trade-off between these two objectives and limiting the full potential of diverse client data. To address this challenge, we propose a novel framework that simultaneously enhances global generalization and local personalization by Rethinking Information Representation in the Federated learning process (FedRIR). Specifically, we introduce Masked Client-Specific Learning (MCSL), which isolates and extracts fine-grained client-specific features tailored to each client's unique data characteristics, thereby enhancing personalization. Concurrently, the Information Distillation Module (IDM) refines the global shared features by filtering out redundant client-specific information, resulting in a purer and more robust global representation that enhances generalization. By integrating the refined global features with the isolated client-specific features, we construct enriched representations that effectively capture both global patterns and local nuances, thereby improving the performance of downstream tasks on the client. The code is available at https://github.com/Deep-Imaging-Group/FedRIR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。