arXiv:2601.13357cs.LGcs.CL2026-01被引 1

厘清状态空间模型与隐马尔可夫模型的异同,连接经典理论与现代NLP模型

On the Relation of State Space Models and Hidden Markov Models

  • 从概率图模型视角对比四类模型的结构与推断机制
  • 揭示线性高斯SSM与卡尔曼滤波的等价性,及与HMM的本质差异
  • 帮助理解Mamba等新模型如何继承经典框架的精髓

状态空间模型(SSMs)和隐马尔可夫模型(HMMs)是建模带隐变量序列数据的基础框架,广泛应用于信号处理、控制论和机器学习。尽管二者具有相似的时间结构,但在隐状态性质、概率假设、推断方法和训练范式上存在根本差异。近期,确定性状态空间模型通过S4和Mamba等架构在自然语言处理中重新兴起,引发了对经典概率SSM、HMM与现代神经序列模型之间关系的新思考。本文系统比较了HMM、线性高斯状态空间模型、卡尔曼滤波以及当代NLP状态空间模型,通过概率图模型视角分析其形式化表达,考察前向-后向推断与卡尔曼滤波等算法,并对比期望最大化(EM)与基于梯度的优化学习流程。通过强调结构相似性与语义差异,阐明了这些模型在何种情况下等价、何时本质不同,以及现代NLP SSM如何关联经典概率模型。分析融合了控制论、概率建模与现代深度学习的多重视角。

原文摘要 · Abstract (English)

State Space Models (SSMs) and Hidden Markov Models (HMMs) are foundational frameworks for modeling sequential data with latent variables and are widely used in signal processing, control theory, and machine learning. Despite their shared temporal structure, they differ fundamentally in the nature of their latent states, probabilistic assumptions, inference procedures, and training paradigms. Recently, deterministic state space models have re-emerged in natural language processing through architectures such as S4 and Mamba, raising new questions about the relationship between classical probabilistic SSMs, HMMs, and modern neural sequence models. In this paper, we present a unified and systematic comparison of HMMs, linear Gaussian state space models, Kalman filtering, and contemporary NLP state space models. We analyze their formulations through the lens of probabilistic graphical models, examine their inference algorithms -- including forward-backward inference and Kalman filtering -- and contrast their learning procedures via Expectation-Maximization and gradient-based optimization. By highlighting both structural similarities and semantic differences, we clarify when these models are equivalent, when they fundamentally diverge, and how modern NLP SSMs relate to classical probabilistic models. Our analysis bridges perspectives from control theory, probabilistic modeling, and modern deep learning.

状态空间模型隐马尔可夫概率建模NLP

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。