提出新方法提升去中心化微调稳定性，收敛更快更准。
ADF-LoRA: Alternating Low-Rank Aggregation for Decentralized Federated Fine-Tuning
- 每轮只同步一个低秩矩阵，混合双矩阵保持状态一致
- 在多个GLUE任务上收敛更快，平均准确率最高
- 适合无服务器架构的分布式模型微调场景
本文重新审视用于联邦微调的交替低秩更新机制，并研究其在去中心化联邦学习（DFL）中的表现。尽管交替更新已在集中式联邦学习中证明可稳定聚合，但在去中心化的点对点通信中，由于相位状态不匹配和客户端间块状发散，带来新挑战。为此提出ADF-LoRA，每轮仅同步一个低秩矩阵，并混合两个矩阵以维持去中心化传播下的参数状态一致性。该设计保留了交替更新的交叉项抑制效果，同时提升了无服务器拓扑下的稳定性。在标准光滑性假设下提供收敛性分析，并在多个GLUE任务上进行评估。实验表明，ADF-LoRA实现更快更平滑的收敛，各项任务平均准确率最高，显著优于现有LoRA变体。
原文摘要 · Abstract (English)
This paper revisits alternating low-rank updates for federated fine-tuning and examines their behavior in decentralized federated learning (DFL). While alternating the LoRA matrices has been shown to stabilize aggregation in centralized FL, extending this mechanism to decentralized, peer-to-peer communication introduces new challenges due to phase-state mismatch and block-wise divergence across clients. We introduce ADF-LoRA, which synchronizes the update of only one low-rank matrix per round and mixes both matrices to maintain more consistent parameter states under decentralized propagation. This design preserves the cross-term suppression effect of alternating updates while improving stability in serverless topologies. We provide a convergence analysis under standard smoothness assumptions and evaluate ADF-LoRA on multiple GLUE tasks. Experiments show that ADF-LoRA achieves faster and smoother convergence and delivers the highest average accuracy across tasks, outperforming existing LoRA variants in decentralized FL by a consistent margin.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。