arXiv:2504.05167cs.LGcs.AI2025-04

用强化学习优化贝叶斯网络结构学习,更高效找最优解。

RLBayes: a Bayesian Network Structure Learning Algorithm via Reinforcement Learning-Based Search Strategy

  • 基于Q-learning构建动态Q表,引导搜索路径。
  • 理论证明可收敛到全局最优结构,实验优于多数启发式算法。
  • 适合需要高精度结构学习的科研与工业场景。

基于评分的贝叶斯网络(BN)结构学习是学习此类概率图模型的有效方法,被广泛认为是不确定性推理中最具说服力的模型之一。然而,随着变量数量增加,结构搜索空间呈超指数增长,使BN结构学习成为NP难问题及组合优化问题(COP)。尽管已有诸多启发式方法取得成功,但结果仍不理想。受Q-learning启发,本文提出一种基于强化学习(RL-based)搜索策略的贝叶斯网络结构学习算法——RLBayes。该方法借鉴强化学习思想,通过动态维护的Q表记录并引导学习过程。通过构建和维护动态Q表,RLBayes在有限空间内实现对无限搜索空间的存储,从而完成基于Q-learning的贝叶斯网络结构学习。不仅理论上证明了RLBayes可收敛至全局最优贝叶斯网络结构,实验也表明其性能优于几乎所有其他启发式搜索算法。

原文摘要 · Abstract (English)

The score-based structure learning of Bayesian network (BN) is an effective way to learn BN models, which are regarded as some of the most compelling probabilistic graphical models in the field of representation and reasoning under uncertainty. However, the search space of structure learning grows super-exponentially as the number of variables increases, which makes BN structure learning an NP-hard problem, as well as a combination optimization problem (COP). Despite the successes of many heuristic methods on it, the results of the structure learning of BN are usually unsatisfactory. Inspired by Q-learning, in this paper, a Bayesian network structure learning algorithm via reinforcement learning-based (RL-based) search strategy is proposed, namely RLBayes. The method borrows the idea of RL and tends to record and guide the learning process by a dynamically maintained Q-table. By creating and maintaining the dynamic Q-table, RLBayes achieve storing the unlimited search space within limited space, thereby achieving the structure learning of BN via Q-learning. Not only is it theoretically proved that RLBayes can converge to the global optimal BN structure, but also it is experimentally proved that RLBayes has a better effect than almost all other heuristic search algorithms.

贝叶斯网络强化学习结构学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。