arXiv:2411.02788cs.ROcs.AI2024-11被引 3

让机器人学会在何时定位,平衡效率与安全风险。

When to Localize? A Risk-Constrained Reinforcement Learning Approach

  • 用强化学习自动判断何时该定位,减少不必要的定位次数。
  • 在未见过环境中成功率提升至少26%。
  • 适合资源受限的水下或空间探索机器人使用。

在标准导航流程中,机器人每一步都进行定位以降低误差。但在某些场景下,获取观测代价高昂,例如水下机器人频繁浮出定位会妨碍其在水下搜寻关键目标(如飞机黑匣子)。若从不定位,状态估计偏差可能导致误入危险区域或遗漏目标区域。为此,本文研究如何让机器人自主决定“何时定位”。将问题建模为双目标优化:最小化定位次数,同时确保失败概率(碰撞或未达目标)在可控范围内。此前工作将其转化为约束部分可观马尔可夫决策过程(POMDP),但求解过慢且需完整模型知识。本文提出RiskRL框架,采用粒子滤波与循环软演员-评论家网络,学习在满足失败概率约束前提下最小化定位次数的策略。实验表明,该方法在未知测试环境中的成功率至少提升26%。

原文摘要 · Abstract (English)

In a standard navigation pipeline, a robot localizes at every time step to lower navigational errors. However, in some scenarios, a robot needs to selectively localize when it is expensive to obtain observations. For example, an underwater robot surfacing to localize too often hinders it from searching for critical items underwater, such as black boxes from crashed aircraft. On the other hand, if the robot never localizes, poor state estimates cause failure to find the items due to inadvertently leaving the search area or entering hazardous, restricted areas. Motivated by these scenarios, we investigate approaches to help a robot determine "when to localize?" We formulate this as a bi-criteria optimization problem: minimize the number of localization actions while ensuring the probability of failure (due to collision or not reaching a desired goal) remains bounded. In recent work, we showed how to formulate this active localization problem as a constrained Partially Observable Markov Decision Process (POMDP), which was solved using an online POMDP solver. However, this approach is too slow and requires full knowledge of the robot transition and observation models. In this paper, we present RiskRL, a constrained Reinforcement Learning (RL) framework that overcomes these limitations. RiskRL uses particle filtering and recurrent Soft Actor-Critic network to learn a policy that minimizes the number of localizations while ensuring the probability of failure constraint is met. Our numerical experiments show that RiskRL learns a robust policy that leads to at least a 26% increase in success rates when traversing unseen test environments.

强化学习主动定位机器人导航风险约束

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。