arXiv:2607.12172cs.DCcs.LG2026-07

分析去中心化优化的资源消耗,揭示不同阶段的瓶颈因素

Decentralized Gradient Descent: Bottleneck Regimes and Budget Complexity

论文配图:Decentralized Gradient Descent: Bottleneck Regimes and Budget Complexity
图 1 · 摘自论文原文
  • 提出瓶颈主导框架,识别初始化、网络连通性等关键影响因素
  • 定义DNR与GCR比值,量化通信与计算资源需求
  • 给出最优步长策略和预算复杂度边界,适合系统设计者参考

去中心化梯度下降(DGD)广泛用于网络中代理间的分布式优化问题。尽管其收敛性已被充分理解,但对达到预定精度所需的通信与计算资源仍知之甚少。本文从资源感知视角研究DGD,刻画达成目标误差水平所需的通信-计算预算。我们构建了一个以瓶颈为核心的框架,揭示在不同误差尺度下,初始化、目标异质性、网络连通性、梯度噪声与通信噪声等因素分别主导优化动态。为此,引入两个基础量:梯度多样性与网络连通性比(DNR)和梯度与通信噪声比(GCR)。我们证明这些量决定了优化过程中遇到的瓶颈序列及相应的预算最优操作策略。通过多阶段分析,推导出最优步长选择和显式的预算复杂度界,定量描述达成指定精度所需的资源。结果揭示了整体预算如何分解为各瓶颈阶段的贡献,并深入揭示了目标异质性、网络连通性、梯度噪声与通信噪声之间的根本权衡。

原文摘要 · Abstract (English)

Decentralized gradient descent (DGD) is widely used for solving distributed optimization problems over networks of agents. While its convergence properties are well understood, less is known about the communication and computation resources required to attain a prescribed accuracy. In this paper, we study DGD from a resource-aware perspective and characterize the communication-computation budget required to attain a target error level. We develop a bottleneck-centric framework in which different factors dominate the optimization dynamics at different error scales. Specifically, we identify operating regimes governed by initialization, objective heterogeneity and network connectivity, gradient noise, and communication noise. To capture these effects, we introduce two fundamental quantities: the gradient-Diversity-to-Network-connectivity Ratio (DNR) and the Gradient-to-Communication-noise Ratio (GCR). We show that these quantities determine the sequence of bottlenecks encountered during optimization and the corresponding budget-optimal operating strategy. Using a multi-stage analysis, we derive optimal stepsize selections and explicit budget-complexity bounds that quantify the budget resources required to attain a prescribed accuracy. The resulting expressions reveal how the overall budget decomposes into contributions associated with successive bottlenecks and provide insight into the fundamental tradeoffs among objective heterogeneity, network connectivity, gradient noise, and communication noise.

去中心化优化资源效率梯度下降系统设计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。