解决联邦学习中长尾分布下的域泛化难题
FEDTAIL: Federated Long-Tailed Domain Generalization with Sharpness-Guided Gradient Matching
- 通过梯度一致性正则缓解分类与对抗目标冲突
- 动态加权策略提升尾部类别的识别性能
- 适合存在标签不平衡和数据分布差异的场景
域泛化旨在训练出在未见目标域上表现稳定的模型,且训练时不访问目标数据。尽管近期平滑损失景观的方法提升了泛化能力,但现有方法在长尾类别分布和矛盾优化目标下表现不佳。我们提出 FedTAIL,一种基于尖锐度引导梯度匹配的联邦域泛化框架。该方法引入梯度一致性正则,缓解分类与对抗目标间的冲突,实现更稳定的收敛;为应对类别不平衡,采用类别级尖锐度最小化,并提出曲率感知的动态加权机制,自适应增强尾部类别的权重;同时,在熵正则化中融入尖锐度感知扰动,强化条件分布对齐,提升域偏移下的鲁棒性。FedTAIL 将优化协调、类别感知正则与条件对齐统一于可扩展的联邦兼容框架中。在多个标准域泛化基准上的广泛评估表明,无论在集中式还是联邦设置下,该方法均达到最优性能,尤其在域偏移和标签不平衡场景下表现突出。
原文摘要 · Abstract (English)
Domain Generalization (DG) seeks to train models that perform reliably on unseen target domains without access to target data during training. While recent progress in smoothing the loss landscape has improved generalization, existing methods often falter under long-tailed class distributions and conflicting optimization objectives. We introduce FedTAIL, a federated domain generalization framework that explicitly addresses these challenges through sharpness-guided, gradient-aligned optimization. Our method incorporates a gradient coherence regularizer to mitigate conflicts between classification and adversarial objectives, leading to more stable convergence. To combat class imbalance, we perform class-wise sharpness minimization and propose a curvature-aware dynamic weighting scheme that adaptively emphasizes underrepresented tail classes. Furthermore, we enhance conditional distribution alignment by integrating sharpness-aware perturbations into entropy regularization, improving robustness under domain shift. FedTAIL unifies optimization harmonization, class-aware regularization, and conditional alignment into a scalable, federated-compatible framework. Extensive evaluations across standard domain generalization benchmarks demonstrate that FedTAIL achieves state-of-the-art performance, particularly in the presence of domain shifts and label imbalance, validating its effectiveness in both centralized and federated settings. Code: https://github.com/sunnyinAI/FedTail
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。