提出可动态网络下无偏采样的去中心化贝叶斯学习算法
DIGing--SGLD: Decentralized and Scalable Langevin Sampling over Time--Varying Networks
- 结合梯度追踪与Langevin采样,支持动态网络上的分布式学习
- 首次给出非渐近的Wasserstein收敛保证,误差在O(√η)邻域内
- 适用于数据分散、网络频繁变化的智能系统,如物联网和边缘计算
从训练数据诱导的目标分布中采样是贝叶斯学习的核心,随机梯度Langevin动力学(SGLD)是实现可扩展后验采样的关键工具,其去中心化变体可在数据分布在多代理网络时实现学习。本文提出DIGing-SGLD,一种专为时变网络上多代理系统设计的去中心化SGLD算法,用于可扩展的贝叶斯学习。现有去中心化SGLD方法仅限于静态拓扑,且多数即使使用全批量仍存在由网络效应引起的稳态采样偏差。DIGing-SGLD通过融合基于Langevin的采样与原用于时变网络去中心化优化的DIGing梯度追踪机制,实现了无需中心协调器的高效无偏采样。据我们所知,这是首个针对时变网络上基于SGLD的去中心化采样提供有限时间非渐近Wasserstein收敛保证的方法,包含明确常数。在标准强凸性和光滑性假设下,DIGing-SGLD以几何速率收敛至目标分布的O(√η)邻域,其中η为步长,对目标精度的依赖与使用固定步长的集中式及静态网络SGLD算法的最佳已知率一致。在贝叶斯线性和逻辑回归上的数值实验验证了理论结果,并展示了DIGing-SGLD在动态网络条件下的强大经验性能。
原文摘要 · Abstract (English)
Sampling from a target distribution induced by training data is central to Bayesian learning, with Stochastic Gradient Langevin Dynamics (SGLD) serving as a key tool for scalable posterior sampling and decentralized variants enabling learning when data are distributed across a network of agents. This paper introduces DIGing-SGLD, a decentralized SGLD algorithm designed for scalable Bayesian learning in multi-agent systems operating over time-varying networks. Existing decentralized SGLD methods are restricted to static network topologies, and many exhibit steady-state sampling bias caused by network effects, even when full batches are used. DIGing-SGLD overcomes these limitations by integrating Langevin-based sampling with the gradient-tracking mechanism of the DIGing algorithm, originally developed for decentralized optimization over time-varying networks, thereby enabling efficient and bias-free sampling without a central coordinator. To our knowledge, we provide the first finite-time non-asymptotic Wasserstein convergence guarantees for decentralized SGLD-based sampling over time-varying networks, with explicit constants. Under standard strong convexity and smoothness assumptions, DIGing-SGLD achieves geometric convergence to an $O(\sqrtη)$ neighborhood of the target distribution, where $η$ is the stepsize, with dependence on the target accuracy matching the best-known rates for centralized and static-network SGLD algorithms using constant stepsize. Numerical experiments on Bayesian linear and logistic regression validate the theoretical results and demonstrate the strong empirical performance of DIGing-SGLD under dynamically evolving network conditions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。