arXiv:2510.21107cs.LGcs.AI2025-10NeurIPS被引 1

提出ESCORT框架,精准建模高维多模态不确定性的信念分布。

ESCORT: Efficient Stein-variational and Sliced Consistency-Optimized Temporal Belief Representation for POMDPs

  • 基于粒子的信念表示,融合相关性感知投影与时间一致性约束。
  • 在高维多模态分布上优于现有方法,提升决策质量与信念精度。
  • 适合复杂动态环境中的强化学习与不确定性决策任务。

在部分可观测马尔可夫决策过程(POMDPs)中,维护和更新对潜在状态的信念分布为不确定条件下的有效决策提供了理论基础。随着环境趋于真实,信念分布变得复杂,标准数学模型难以准确捕捉,导致表征失准。尽管深度学习与概率建模有所进展,现有信念近似方法仍无法有效表示高维、多模态的不确定性结构,引发估计误差并导致次优行为。为此,我们提出ESCORT(高效斯坦因变分与切片一致性优化的时间信念表征),一种基于粒子的框架,用于捕捉高维信念空间中的复杂多模态分布。ESCORT在SVG(Stein变分梯度下降)基础上引入两项创新:相关性感知投影以建模状态维度间的依赖关系,以及时间一致性约束以稳定更新并保持相关结构。该方法保留了吸引-排斥粒子动力学,同时实现对复杂相关模式的精确建模。相比易退化或受分布假设限制的粒子滤波器与参数化方法,ESCORT能动态适应信念景观复杂度,无需重采样。我们在POMDP领域与不同维度的合成多模态分布上进行了广泛评估,结果表明其在信念近似精度与下游决策质量方面持续优于当前最优方法。

原文摘要 · Abstract (English)

In Partially Observable Markov Decision Processes (POMDPs), maintaining and updating belief distributions over possible underlying states provides a principled way to summarize action-observation history for effective decision-making under uncertainty. As environments grow more realistic, belief distributions develop complexity that standard mathematical models cannot accurately capture, creating a fundamental challenge in maintaining representational accuracy. Despite advances in deep learning and probabilistic modeling, existing POMDP belief approximation methods fail to accurately represent complex uncertainty structures such as high-dimensional, multi-modal belief distributions, resulting in estimation errors that lead to suboptimal agent behaviors. To address this challenge, we present ESCORT (Efficient Stein-variational and sliced Consistency-Optimized Representation for Temporal beliefs), a particle-based framework for capturing complex, multi-modal distributions in high-dimensional belief spaces. ESCORT extends SVGD with two key innovations: correlation-aware projections that model dependencies between state dimensions, and temporal consistency constraints that stabilize updates while preserving correlation structures. This approach retains SVGD's attractive-repulsive particle dynamics while enabling accurate modeling of intricate correlation patterns. Unlike particle filters prone to degeneracy or parametric methods with fixed representational capacity, ESCORT dynamically adapts to belief landscape complexity without resampling or restrictive distributional assumptions. We demonstrate ESCORT's effectiveness through extensive evaluations on both POMDP domains and synthetic multi-modal distributions of varying dimensionality, where it consistently outperforms state-of-the-art methods in terms of belief approximation accuracy and downstream decision quality.

POMDP信念表示多模态粒子方法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。