arXiv:2506.07578cs.LGcs.AI2025-06中稿 · ECSQARU 2025, exte…

用最可能的转移路径加速时间推演,大幅降噪提速

Denoising the Future: Top-p Distributions for Moving Through Time

  • 只保留概率累积值前p的转移路径,跳过低概率状态
  • 速度提升超10倍,总变差距离误差低于0.09
  • 适合对推理效率要求高、可容忍微小误差的场景

动态概率模型的推理涉及昂贵计算。以隐马尔可夫模型为例,时间推进需遍历整个状态空间,即使概率极低的状态也参与计算,导致效率低下且噪声增加。本文提出仅保留累积概率为p的前p个最可能转移路径,实现“去噪未来”,显著提升推理效率。理论证明:使用顶-p转移带来的误差受p和模型最小混合率的双重约束。实验证明,该方法可实现至少一个数量级的速度提升,总变差距离误差低于0.09;而仅保留顶-p状态的方法因需遍历所有状态,效率更低,且误差未必降低。虽然顶-p转移极具潜力,但顶-p状态不推荐。

原文摘要 · Abstract (English)

Inference in dynamic probabilistic models is a complex task involving expensive operations. In particular, for Hidden Markov Models, the whole state space has to be enumerated for advancing in time. Even states with negligible probabilities are considered, resulting in computational inefficiency and possibly increased noise due to the propagation of unlikely probability mass. We propose to denoise the future and speed up inference by using only the top-p transitions, i.e., the most probable transitions with accumulated probability p. We show that the error introduced by using only the top-p transitions is bound by $p$ and the so-called minimal mixing rate of the underlying model. We also show the same bound when using only the top-p states, which is the same, just for the states. Moreover, in our empirical evaluation, we show that we can, when using top-p transitions, expect speedups of at least an order of magnitude, while the error in terms of total variation distance is below 0.09. Using the top-p states is slower than top-p transitions since we iterate over all states in each time step and sometimes lead empirically to a higher error. With a more sophisticated implementation, the speed-up, if any, would be really small. While top-p transitions look really promising, we cannot recommend top-p states and discuss why it is of the slower, while the error does not necessarily decrease.

推理加速隐马尔可夫概率模型高效采样

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。