受大脑混沌学习启发,提出新型图神经网络训练方法,显著提升大规模组合优化性能。
Brain-inspired Chaotic Graph Backpropagation for Large-scale Combinatorial Optimization
- 基于脑启发的混沌动力学设计新反向传播算法,避免陷入局部最优。
- 在最大独立集、最大割等任务上超越现有GNN方法及顶尖优化算法。
- 可作为通用模块插入任意GNN框架,提升模型全局搜索能力。
图神经网络(GNN)通过无监督学习可高效求解大规模组合优化问题(COP),具备广泛应用潜力。然而,当前主流基于反向传播的训练算法易陷入局部极小值,导致优化性能低于现有最先进(SOTA)方法。受真实大脑学习可能具有混沌动力学的启发,本文提出混沌图反向传播(CGBP)算法,在GNN中引入局部损失函数,使训练过程兼具混沌性与高效率。不同于现有方法,我们证明此类混沌动态的全局遍历性和伪随机性,使CGBP能有效且全局地学习每个最优GNN,从而高效求解COP。实验在多个大规模基准数据集上验证了该方法在最大独立集、最大割和图着色等任务上的优越性,不仅优于其他GNN算法,也超越了当前SOTA方法。此外,CGBP作为通用的GNN学习算法,可作为插件模块集成到任意现有方法中,以提升性能。
原文摘要 · Abstract (English)
Graph neural networks (GNNs) with unsupervised learning can solve large-scale combinatorial optimization problems (COPs) with efficient time complexity, making them versatile for various applications. However, since this method maps the combinatorial optimization problem to the training process of a graph neural network, and the current mainstream backpropagation-based training algorithms are prone to fall into local minima, the optimization performance is still inferior to the current state-of-the-art (SOTA) COP methods. To address this issue, inspired by possibly chaotic dynamics of real brain learning, we introduce a chaotic training algorithm, i.e. chaotic graph backpropagation (CGBP), which introduces a local loss function in GNN that makes the training process not only chaotic but also highly efficient. Different from existing methods, we show that the global ergodicity and pseudo-randomness of such chaotic dynamics enable CGBP to learn each optimal GNN effectively and globally, thus solving the COP efficiently. We have applied CGBP to solve various COPs, such as the maximum independent set, maximum cut, and graph coloring. Results on several large-scale benchmark datasets showcase that CGBP can outperform not only existing GNN algorithms but also SOTA methods. In addition to solving large-scale COPs, CGBP as a universal learning algorithm for GNNs, i.e. as a plug-in unit, can be easily integrated into any existing method for improving the performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。