提出OvA-LP框架，有效解决联邦微调中非独立同分布数据的漂移问题。
OvA-LP: A Simple and Efficient Framework for Federated Learning on Non-IID Data
- 通过冻结编码器+一对多分类头，从源头抑制本地更新漂移
- 在非独立同分布条件下保留95.9%的基准准确率，显著优于现有方法
- 适合资源受限场景，且对标签噪声具有强鲁棒性
联邦微调（FFT）将预训练模型适配到分布式数据，但在客户端分布异质时易受局部漂移影响，导致全局模型出现系统性偏差和方差放大。现有聚合与个性化方法多为事后修正，极端非独立同分布下表现脆弱。我们提出OvA-LP，据知是首个在基于参数高效微调（PEFT）的联邦微调范式中，从源头抑制漂移的极简框架。该框架结合冻结编码器的线性探测与一对多分类头，采用两阶段流程，保持预训练特征几何结构并解耦输出概率，从而阻断漂移放大机制。在100个客户端的CIFAR-100数据集上，三种划分方式（shard-1、shard-2、Bernoulli-Dirichlet）平均下，OvA-LP保留95.9%的独立同分布准确率，而最先进的基线仅保留10.1%（PFPT）和34.5%（FFT-MoE）。此外，该方法对对称与非对称标签噪声均具鲁棒性。预计算编码器特征使每轮开销几乎与编码器规模无关。结果表明，OvA-LP为异质环境下的鲁棒联邦微调提供了原理清晰且高效的解决方案。
原文摘要 · Abstract (English)
Federated fine-tuning (FFT) adapts foundation models to decentralized data but remains fragile under heterogeneous client distributions due to local drift, i.e., client-level update divergences that induce systematic bias and amplified variance in the global model. Existing aggregation and personalization methods largely correct drift post hoc, which proves brittle under extreme non-IID conditions. We introduce OvA-LP, a minimalist framework that is, to our knowledge, the first explicitly designed to suppress drift at its source within the PEFT-based FFT paradigm. OvA-LP combines linear probing on a frozen encoder with a one-vs-all head and a simple two-stage procedure, preserving pretrained feature geometry and decoupling logits to prevent the mechanisms that amplify drift. On CIFAR-100 with 100 clients, averaged over shard-1, shard-2, and Bernoulli-Dirichlet partitions, OvA-LP retains 95.9% of its IID accuracy, whereas state-of-the-art FFT baselines retain only 10.1% (PFPT) and 34.5% (FFT-MoE) under the same conditions. OvA-LP further maintains resilience under both symmetric and asymmetric label noise. In addition, precomputing encoder features makes per-round cost nearly independent of encoder size. Together, these results demonstrate that OvA-LP provides a principled and efficient basis for robust FFT under heterogeneity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。