用图神经网络学着调优分布式ADMM的收敛速度。
Learning to accelerate distributed ADMM using graph neural networks
- 用图神经网络预测自适应步长和通信权重
- 训练后收敛速度更快，解的质量更高
- 适合需要快速求解分布式优化问题的研究者
分布式优化是大规模机器学习与控制应用的基础。其中，交替方向乘子法（ADMM）因具备强收敛保证且适用于去中心化计算而广受关注。然而，标准ADMM存在收敛慢、对超参数敏感的问题。本文发现分布式ADMM迭代可自然嵌入图神经网络（GNN）的消息传递框架中。基于此，我们提出一种利用GNN学习自适应步长和通信权重的方法，根据当前迭代状态预测超参数。通过固定步数的ADMM展开训练，网络端到端最小化特定问题类在给定计算预算下的解距离，同时保持算法收敛性。数值实验表明，所提方法在训练预算内及之外均显著提升收敛速度与解质量。代码已开源：https://github.com/paulhausner/learning-distributed-admm。
原文摘要 · Abstract (English)
Distributed optimization is fundamental to large-scale machine learning and control applications. Among existing methods, the alternating direction method of multipliers (ADMM) has gained popularity due to its strong convergence guarantees and suitability for decentralized computation. However, ADMM can suffer from slow convergence and high sensitivity to hyperparameter choices. In this work, we show that distributed ADMM iterations can be naturally expressed within the message-passing framework of graph neural networks (GNNs). Building on this connection, we propose learning adaptive step sizes and communication weights through a GNN that predicts these yperparameters based on the current iterates. By unrolling ADMM for a fixed number of iterations, we train the network end-to-end to minimize the solution distance after these iterations for a given problem class, while preserving the algorithm's convergence properties. Numerical experiments demonstrate that our learned variant consistently improves convergence speed and solution quality compared to standard ADMM, both within the trained computational budget and beyond. The code is available at https://github.com/paulhausner/learning-distributed-admm.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。