arXiv:2410.07630cs.RO2024-10中稿 · ISRR 2024

通过新观测空间简化POMDP规划,实现快速决策并保证效果。

Simplified POMDP Planning with an Alternative Observation Space and Formal Performance Guarantees

  • 用更简洁的观测空间替代原模型,降低计算复杂度。
  • 推导出简化后与原始POMDP之间的最优价值函数边界。
  • 适合需要实时性与可靠性保障的机器人决策场景。

在部分可观测环境中进行在线规划是机器人与人工智能的核心能力。部分可观测马尔可夫决策过程(POMDP)为此类挑战性问题提供了数学上严谨的框架,但求解最优策略计算成本高,仅适用于小规模问题。本文提出一种新方法:通过切换到更紧凑的替代观测空间和简化模型来加速规划,并提供形式化性能保证。我们引入信念树拓扑概念,编码树中使用原始与替代观测空间及模型的层级与分支结构。每种拓扑对应特定策略空间与规划性能。关键贡献在于推导出原始POMDP最优Q函数与给定拓扑下简化树之间的上下界,用于在不同拓扑间动态调整直至确定原始问题的最优动作。进一步地,我们实例化该框架,其中替代观测空间对应状态完全可观情况。在仿真中评估了该方法,对比精确与近似POMDP求解器,显著提升速度同时保持解的质量。我们认为本工作为带形式保证的在线POMDP规划开辟了新路径。

原文摘要 · Abstract (English)

Online planning under uncertainty in partially observable domains is an essential capability in robotics and AI. The partially observable Markov decision process (POMDP) is a mathematically principled framework for addressing decision-making problems in this challenging setting. However, finding an optimal solution for POMDPs is computationally expensive and is feasible only for small problems. In this work, we contribute a novel method to simplify POMDPs by switching to an alternative, more compact, observation space and simplified model to speedup planning with formal performance guarantees. We introduce the notion of belief tree topology, which encodes the levels and branches in the tree that use the original and alternative observation space and models. Each belief tree topology comes with its own policy space and planning performance. Our key contribution is to derive bounds between the optimal Q-function of the original POMDP and the simplified tree defined by a given topology with a corresponding simplified policy space. These bounds are then used as an adaptation mechanism between different tree topologies until the optimal action of the original POMDP can be determined. Further, we consider a specific instantiation of our framework, where the alternative observation space and model correspond to a setting where the state is fully observable. We evaluate our approach in simulation, considering exact and approximate POMDP solvers and demonstrating a significant speedup while preserving solution quality. We believe this work opens new exciting avenues for online POMDP planning with formal performance guarantees.

POMDP在线规划机器人决策性能保证

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。