从动态轨迹中学习高阶马尔可夫随机场,突破了传统独立采样下的计算瓶颈。
Bypassing the Noisy Parity Barrier: Learning Higher-Order Markov Random Fields from Dynamics
- 利用格劳伯动力学生成的轨迹数据,直接学习高阶马尔可夫随机场。
- 仅需 $\widetilde{O}_k(n)$ 次更新即可在 $\widetilde{O}_k(n^2)$ 时间内恢复图结构与参数。
- 相比传统方法的 $n^{Θ(k)}$ 下界,效率显著提升,适合真实动态系统建模。
我们研究从时间相关样本中学习图形模型(即马尔可夫随机场,MRF)的问题。传统统计方法通常假设独立同分布(i.i.d.)样本,但自然界观测往往由随机过程演化而来。在计算上,即使生成一个真实MRF样本也难以实现,除非 $\mathsf{NP} = \mathsf{RP}$;且所有基于i.i.d.样本的学习算法都面临来自带噪声奇偶性问题的下界,导致运行时间极长。本文首次证明:来自格劳伯动力学的轨迹数据能突破这些根本性计算下界。对于一个 $k$ 阶MRF,只需 $\widetilde{O}_k(n)$ 次站点更新,即可在 $\widetilde{O}_k(n^2)$ 时间内恢复其图结构与参数。相比之下,以往方法在稀疏情形下仍需 $n^{Θ(k)}$ 时间,而本工作实现了指数级加速。结果表明,更贴近现实的动态模型反而带来更高的学习效率,颠覆了传统认知。
原文摘要 · Abstract (English)
We consider the problem of learning graphical models, also known as Markov random fields (MRFs) from temporally correlated samples. As in many traditional statistical settings, fundamental results in the area all assume independent samples from the distribution. However, these samples generally will not directly correspond to more realistic observations from nature, which instead evolve according to some stochastic process. From the computational lens, even generating a single sample from the true MRF distribution is intractable unless $\mathsf{NP}=\mathsf{RP}$, and moreover, any algorithm to learn from i.i.d. samples requires prohibitive runtime due to hardness reductions to the parity with noise problem. These computational barriers for sampling and learning from the i.i.d. setting severely lessen the utility of these breakthrough results for this important task; however, dropping this assumption typically only introduces further algorithmic and statistical complexities. In this work, we surprisingly demonstrate that the direct trajectory data from a natural evolution of the MRF overcomes the fundamental computational lower bounds to efficient learning. In particular, we show that given a trajectory with $\widetilde{O}_k(n)$ site updates of an order $k$ MRF from the Glauber dynamics, a well-studied, natural stochastic process on graphical models, there is an algorithm that recovers the graph and the parameters in $\widetilde{O}_k(n^2)$ time. By contrast, all prior algorithms for learning order $k$ MRFs inherently suffer from $n^{Θ(k)}$ runtime even in sparse instances due to the reductions to sparse parity with noise. Our results thus surprisingly show that this more realistic, but intuitively less tractable, model for MRFs actually leads to efficiency far beyond what is known and believed to be true in the traditional i.i.d. case.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。