arXiv:2604.19518cs.LGcs.SY2026-04

去中心化学习能加速模型收敛,比中心化方法更高效。

Accelerating Optimization and Machine Learning through Decentralization

论文配图:Accelerating Optimization and Machine Learning through Decentralization
图 1 · 摘自论文原文
  • 将数据和计算分散到多个设备上进行学习
  • 在逻辑回归与神经网络训练中减少迭代次数达成最优
  • 适合注重隐私与通信受限的高效学习场景

去中心化优化使多个设备在仅访问本地数据集的情况下协同训练全局机器学习模型。相比传统集中式学习需将所有数据汇聚至中心服务器,去中心化方式提升了隐私保护与可扩展性。然而,过去普遍认为去中心化仅是因通信限制或数据隐私问题而被迫采用的折衷方案。本研究发现,去中心化反而可能加速收敛,在达到最优解所需的迭代次数上优于集中式方法,即使每次迭代耗时相同。通过逻辑回归与神经网络训练实例验证,分布式计算可实现比集中处理更快的学习速度。这一发现挑战了长期固有认知,揭示去中心化作为战略优势,为更高效的优化与机器学习提供新机遇。

原文摘要 · Abstract (English)

Decentralized optimization enables multiple devices to learn a global machine learning model while each individual device only has access to its local dataset. By avoiding the need for training data to leave individual users' devices, it enhances privacy and scalability compared to conventional centralized learning, where all data has to be aggregated to a central server. However, decentralized optimization has traditionally been viewed as a necessary compromise, used only when centralized processing is impractical due to communication constraints or data privacy concerns. In this study, we show that decentralization can paradoxically accelerate convergence, outperforming centralized methods in the number of iterations needed to reach optimal solutions. Through examples in logistic regression and neural network training, we demonstrate that distributing data and computation across multiple agents can lead to faster learning than centralized approaches, even when each iteration is assumed to take the same amount of time, whether performed centrally on the full dataset or decentrally on local subsets. This finding challenges longstanding assumptions and reveals decentralization as a strategic advantage, offering new opportunities for more efficient optimization and machine learning.

去中心化优化加速隐私学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。