提出懒惰搜索法,让机器人在复杂环境下更快规划。
Lazy Heuristic Search for Solving POMDPs with Expensive-to-Compute Belief Transitions
- 用估值估计延迟计算昂贵的信念转移,节省计算时间。
- 在复杂操作、崎岖地形等场景中,规划速度提升显著。
- 适合需要频繁推理的机器人任务,如接触式操作与导航。
基于启发式搜索的求解器(如RTDP-Bel和LAO*)在部分可观测马尔可夫决策过程(POMDP)中表现优异,通常将其建模为信念MDP。信念是系统状态的概率分布,从父信念和动作出发,计算信念转移需进行贝叶斯更新,结合转移与观测模型以确定后继信念及其转移概率。然而,在机器人领域存在一类问题,其转移与观测模型涉及高成本物理仿真、射线追踪或碰撞检测,导致信念转移计算极其耗时,严重影响规划效率。为此,本文提出懒惰型RTDP-Bel和懒惰型LAO*,通过利用Q值估计延迟昂贵的信念转移计算,大幅减少规划时间。我们在接触丰富的操作任务(姿态估计)、粗糙地形户外导航及使用一维激光雷达的室内导航等场景中验证了该方法的优越性。此外,还讨论了适用于常见问题类型的实用Q值估计技术。实验表明,该方法在显著提升规划速度的同时,仍能保持高质量解。
原文摘要 · Abstract (English)
Heuristic search solvers like RTDP-Bel and LAO* have proven effective for computing optimal and bounded sub-optimal solutions for Partially Observable Markov Decision Processes (POMDPs), which are typically formulated as belief MDPs. A belief represents a probability distribution over possible system states. Given a parent belief and an action, computing belief state transitions involves Bayesian updates that combine the transition and observation models of the POMDP to determine successor beliefs and their transition probabilities. However, there is a class of problems, specifically in robotics, where computing these transitions can be prohibitively expensive due to costly physics simulations, raycasting, or expensive collision checks required by the underlying transition and observation models, leading to long planning times. To address this challenge, we propose Lazy RTDP-Bel and Lazy LAO*, which defer computing expensive belief state transitions by leveraging Q-value estimation, significantly reducing planning time. We demonstrate the superior performance of the proposed lazy planners in domains such as contact-rich manipulation for pose estimation, outdoor navigation in rough terrain, and indoor navigation with a 1-D LiDAR sensor. Additionally, we discuss practical Q-value estimation techniques for commonly encountered problem classes that our lazy planners can leverage. Our results show that lazy heuristic search methods dramatically improve planning speed by postponing expensive belief transition evaluations while maintaining solution quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。