解决联邦学习中属性偏移导致的误关联问题,提升模型在未知数据上的泛化能力。
Global Intervention and Distillation for Federated Out-of-Distribution Generalization
- 通过全局干预模块解耦图像对象与背景,注入背景信息以打破虚假关联。
- 在三个数据集上显著提升模型对未见样本的主目标关注力,性能优于现有方法。
- 适合需要跨客户端稳定泛化的联邦学习场景,尤其应对数据分布不一致问题。
联邦学习中的属性偏移导致本地模型关注非因果关联,引导优化方向不一致,引发性能下降与收敛不稳定。现有方法多依赖数据增强提升样本多样性或知识蒸馏学习不变表示,但生成数据质量不稳定且缺乏领域信息,限制了其在未见样本上的表现。本文提出全局干预与蒸馏方法FedGID,利用多样化属性特征进行后门调整,打破背景与标签间的虚假关联。该方法包含两个核心模块:全局干预模块自适应地解耦图像中的物体与背景,将背景信息注入随机样本,干预样本分布,使背景与所有类别关联,防止模型将背景-标签关系视为因果;全局蒸馏模块通过统一知识库指导客户端模型的表征学习,避免本地模型过度拟合客户端特有属性。在三个数据集上的实验表明,FedGID显著提升了模型对未见数据中主体的关注能力,在协同建模中优于现有方法。
原文摘要 · Abstract (English)
Attribute skew in federated learning leads local models to focus on learning non-causal associations, guiding them towards inconsistent optimization directions, which inevitably results in performance degradation and unstable convergence. Existing methods typically leverage data augmentation to enhance sample diversity or employ knowledge distillation to learn invariant representations. However, the instability in the quality of generated data and the lack of domain information limit their performance on unseen samples. To address these issues, this paper presents a global intervention and distillation method, termed FedGID, which utilizes diverse attribute features for backdoor adjustment to break the spurious association between background and label. It includes two main modules, where the global intervention module adaptively decouples objects and backgrounds in images, injects background information into random samples to intervene in the sample distribution, which links backgrounds to all categories to prevent the model from treating background-label associations as causal. The global distillation module leverages a unified knowledge base to guide the representation learning of client models, preventing local models from overfitting to client-specific attributes. Experimental results on three datasets demonstrate that FedGID enhances the model's ability to focus on the main subjects in unseen data and outperforms existing methods in collaborative modeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。