用分数阶记忆提升分布式优化速度与稳定性
Fractional Order Distributed Optimization
- 引入分数阶记忆项改进分布式优化
- 在病态问题上收敛速度最快快4倍
- 适合需要快速稳定训练的联邦学习场景
分布式优化是现代机器学习应用（如联邦学习）的基础，但现有方法在病态问题上常面临收敛慢和稳定性-速度权衡。本文提出分数阶分布式优化（FrODO），一种具有理论保障的框架，通过引入分数阶记忆项，显著改善复杂优化景观下的收敛性能。该方法在任意强连通网络下均可实现可证明的线性收敛。实验验证表明，FrODO在病态问题上的收敛速度相比基线最高提升4倍，在联邦神经网络训练中提速2-3倍，同时保持稳定性和理论保证。
原文摘要 · Abstract (English)
Distributed optimization is fundamental to modern machine learning applications like federated learning, but existing methods often struggle with ill-conditioned problems and face stability-versus-speed tradeoffs. We introduce fractional order distributed optimization (FrODO); a theoretically-grounded framework that incorporates fractional-order memory terms to enhance convergence properties in challenging optimization landscapes. Our approach achieves provable linear convergence for any strongly connected network. Through empirical validation, our results suggest that FrODO achieves up to 4 times faster convergence versus baselines on ill-conditioned problems and 2-3 times speedup in federated neural network training, while maintaining stability and theoretical guarantees.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。