提出LP-FT方法，解决联邦学习中个性化与泛化冲突问题
A Closer Look at Personalized Fine-Tuning in Heterogeneous Federated Learning
- 先线性探测再全量微调，分阶段更新参数避免特征扭曲
- 在7个数据集上优于6种主流微调方法，平衡个性化与泛化性能
- 揭示联邦特征扭曲现象，给出适用条件和部署建议
联邦学习（FL）实现去中心化、隐私保护的模型训练，但难以兼顾全局泛化与局部个性化，因客户端数据分布异构。个性化微调（PFT）作为常见后处理方案，常因本地数据偏差导致过拟合或在域偏移下失效。本文将线性探测后全量微调（LP-FT）这一集中式策略引入联邦设置，系统评估其在7个数据集和6种PFT变体上的表现。结果表明，LP-FT能更好平衡个性化与泛化。分析发现，本地微调会引发联邦特征扭曲，而LP-FT通过分阶段参数更新有效缓解该问题。进一步建立了在部分特征重叠、协变量-概念偏移等条件下，LP-FT优于标准微调的理论条件，为鲁棒个性化部署提供可操作指导。
原文摘要 · Abstract (English)
Federated Learning (FL) enables decentralized, privacy-preserving model training but struggles to balance global generalization and local personalization due to non-identical data distributions across clients. Personalized Fine-Tuning (PFT), a popular post-hoc solution, fine-tunes the final global model locally but often overfits to skewed client distributions or fails under domain shifts. We propose adapting Linear Probing followed by full Fine-Tuning (LP-FT), a principled centralized strategy for alleviating feature distortion (Kumar et al., 2022), to the FL setting. Through systematic evaluation across seven datasets and six PFT variants, we demonstrate LP-FT's superiority in balancing personalization and generalization. Our analysis uncovers federated feature distortion, a phenomenon where local fine-tuning destabilizes globally learned features, and theoretically characterizes how LP-FT mitigates this via phased parameter updates. We further establish conditions (e.g., partial feature overlap, covariate-concept shift) under which LP-FT outperforms standard fine-tuning, offering actionable guidelines for deploying robust personalization in FL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。