arXiv:2502.00645cs.DCcs.LG2025-02被引 5

在随机延迟服务器场景下,编码计算可实现高精度近似结果

General Coded Computing in a Probabilistic Straggler Regime

  • 针对概率性延迟服务器,分析两类通用编码计算方案的误差
  • 证明平均误差随服务器数增加以特定速率收敛至零
  • 适用于深度神经网络等复杂任务,理论与实验均验证有效

编码计算在应对分布式计算中的延迟服务器问题上展现出良好效果。然而,大多数现有方案仅适用于精确计算,且依赖于高度结构化的函数。近期出现的通用编码计算方法采用近似计算,允许更多响应结果带来更高精度估计,引入新挑战。本文研究各服务器以概率 $p$ 独立成为延迟服务器这一实际场景。理论上分析了两种通用编码计算方案——BACC 和 LeTCC 的近似误差。结果显示,在该概率模型下,两者的平均近似误差分别以至少 $\mathcal{O}(\log^3_{\frac{1}{p}}(N) \cdot N^{-3})$ 与 $\mathcal{O}(\log^4_{\frac{1}{p}}(N) \cdot N^{-2})$ 的速率收敛至零。尽管平均延迟服务器数量为 $Np$,但由于服务器独立性,误差仍可收敛。实验在多种计算函数(包括深度神经网络)上验证了理论结论。

原文摘要 · Abstract (English)

Coded computing has demonstrated promising results in addressing straggler resiliency in distributed computing systems. However, most coded computing schemes are designed for exact computation, requiring the number of responding servers to exceed a certain recovery threshold. Additionally, these schemes are tailored for highly structured functions. Recently, new coded computing schemes for general computing functions, where exact computation is replaced with approximate computation, have emerged. In these schemes, the availability of additional results corresponds to more accurate estimation of computational tasks. This flexibility introduces new questions that need to be addressed. This paper addresses the practically important scenario in the context of general coded computing, where each server may become a straggler with a probability $p$, independently from others. We theoretically analyze the approximation error of two existing general coded computing schemes: Berrut Approximate Coded Computing (BACC) and Learning Theoretic Coded Computing (LeTCC). Under the probabilistic straggler configuration, we demonstrate that the average approximation error for BACC and LeTCC converge to zero with the rate of at least $\mathcal{O}(\log^3_{\frac{1}{p}}(N)\cdot{N^{-3}})$ and $\mathcal{O}(\log^4_{\frac{1}{p}}(N)\cdot{N^{-2}})$, respectively. This is perhaps surprising, as earlier results does not indicate a convergence when the number of stragglers scales with the total number of servers $N$. However, in this case, despite the average number of stragglers being $Np$, the independence of servers in becoming stragglers allows the approximation error to converge to zero. These theoretical results are validated through experiments on various computing functions, including deep neural networks.

编码计算近似计算分布式系统误差分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。