用深度学习自动构造马尔可夫链的稳定性函数和稳态分布
Deep Learning for Markov Chains: Lyapunov Functions, Poisson's Equation, and Stationary Distributions
- 用神经网络求解基于首次转移分析的积分方程，自动化构造李雅普诺夫函数
- 方法在非紧致状态空间上仍有效，可推广至排队论等实际场景
- 适合对随机系统稳定性分析或稳态建模感兴趣的工程师与研究人员
李雅普诺夫函数是证明马尔可夫模型稳定性的关键工具，但传统构造依赖大量创造性和分析工作。本文提出利用深度学习自动完成这一过程：通过训练神经网络来满足由首次转移分析导出的积分方程。该方法不仅可用于稳定性分析，还可扩展求解泊松方程并估计平稳分布。尽管神经网络通常仅在紧致域上表现良好，本方法在非紧致状态空间的马尔可夫链上依然有效。我们在排队论等多个实例中验证了该方法的有效性。
原文摘要 · Abstract (English)
Lyapunov functions are fundamental to establishing the stability of Markovian models, yet their construction typically demands substantial creativity and analytical effort. In this paper, we show that deep learning can automate this process by training neural networks to satisfy integral equations derived from first-transition analysis. Beyond stability analysis, our approach can be adapted to solve Poisson's equation and estimate stationary distributions. While neural networks are inherently function approximators on compact domains, it turns out that our approach remains effective when applied to Markov chains on non-compact state spaces. We demonstrate the effectiveness of this methodology through several examples from queueing theory and beyond.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。