解决联邦学习中分布外数据的双重挑战,同时提升泛化与检测能力。
FOOGD: Federated Collaboration for Both Out-of-distribution Generalization and Detection
- 基于评分模型估计各客户端分布,无需先验约束
- 可同时检测语义偏移数据并增强对协变量偏移的泛化能力
- 适合存在多种分布外数据的现实联邦学习场景
联邦学习(FL)通过协作客户端模型捕捉全局知识,但在真实场景中因同时存在分布内数据和意外的分布外(OOD)数据(如协变量偏移和语义偏移)而可靠性不足。现有研究通常仅关注协变量偏移的泛化或语义偏移的检测,忽视了两者共存的情况。本文提出FOOGD,通过估计每个客户端的概率密度,获得可靠的全局分布以指导后续联邦过程。FOOGD中的SM3D模型在无先验约束下估计任意分布,并有效检测语义偏移数据;SAG模块则提供不变且多样化的知识,支持本地协变量偏移泛化和客户端性能泛化。实验表明,FOOGD具有三大优势:(1) 可靠估计非归一化去中心化分布,(2) 通过评分值检测语义偏移数据,(3) 通过正则化特征提取器实现对协变量偏移的泛化。项目开源地址:https://github.com/XeniaLLL/FOOGD-main.git。
原文摘要 · Abstract (English)
Federated learning (FL) is a promising machine learning paradigm that collaborates with client models to capture global knowledge. However, deploying FL models in real-world scenarios remains unreliable due to the coexistence of in-distribution data and unexpected out-of-distribution (OOD) data, such as covariate-shift and semantic-shift data. Current FL researches typically address either covariate-shift data through OOD generalization or semantic-shift data via OOD detection, overlooking the simultaneous occurrence of various OOD shifts. In this work, we propose FOOGD, a method that estimates the probability density of each client and obtains reliable global distribution as guidance for the subsequent FL process. Firstly, SM3D in FOOGD estimates score model for arbitrary distributions without prior constraints, and detects semantic-shift data powerfully. Then SAG in FOOGD provides invariant yet diverse knowledge for both local covariate-shift generalization and client performance generalization. In empirical validations, FOOGD significantly enjoys three main advantages: (1) reliably estimating non-normalized decentralized distributions, (2) detecting semantic shift data via score values, and (3) generalizing to covariate-shift data by regularizing feature extractor. The prejoct is open in https://github.com/XeniaLLL/FOOGD-main.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。