通过自适应调制提升联邦学习泛化能力，适配不同客户端数据分布。
Generalizable Federated Learning using Client Adaptive Focal Modulation
- 基于任务感知嵌入动态生成客户端专属调制层，实现个性化适配。
- 在8个异构数据集上超越主流方法，跨任务设置下性能提升显著。
- 低秩超网络设计降低通信开销，适合资源受限场景部署。
联邦学习（FL）在保护隐私的前提下实现了分布式客户端的协同训练。此前工作TransFed提出基于Transformer的鲁棒框架，利用可学习的超网络为每个客户端生成个性化焦点调制层，在非同质分布（non-IID）和跨域场景中表现优异。本文进一步提出AdaptFED，深化对焦点调制在泛化联邦学习中的应用：（1）引入任务感知客户端嵌入，优化调制动态的个性化；（2）给出改进的适配性能理论边界；（3）在时间序列与多语言等更多模态上验证有效性。同时，我们提出TransFed高效变体，通过低秩超网络条件控制降低服务器-客户端通信开销，支持资源受限环境下的可扩展部署。在8个多样化数据集上的实验表明，该方法在无源和跨任务联邦设置中显著优于现有基线。研究结果不仅拓展了焦点调制在联邦学习中的能力，也为更自适应、可扩展、泛化的基于Transformer的联邦系统铺平道路。代码已公开于http://github.com/Tajamul21/TransFed。
原文摘要 · Abstract (English)
Federated learning (FL) has proven essential for privacy-preserving, collaborative training across distributed clients. Our prior work, TransFed, introduced a robust transformer-based FL framework that leverages a learn-to-adapt hypernetwork to generate personalized focal modulation layers per client, outperforming traditional methods in non-IID and cross-domain settings. In this extended version, we propose AdaptFED, where we deepen the investigation of focal modulation in generalizable FL by incorporating: (1) a refined adaptation strategy that integrates task-aware client embeddings to personalize modulation dynamics further, (2) enhanced theoretical bounds on adaptation performance, and (3) broader empirical validation across additional modalities, including time-series and multilingual data. We also introduce an efficient variant of TransFed that reduces server-client communication overhead via low-rank hypernetwork conditioning, enabling scalable deployment in resource-constrained environments. Extensive experiments on eight diverse datasets reaffirm the superiority of our method over state-of-the-art baselines, particularly in source-free and cross-task federated setups. Our findings not only extend the capabilities of focal modulation in FL but also pave the way for more adaptive, scalable, and generalizable transformer-based federated systems. The code is available at http://github.com/Tajamul21/TransFed
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。