提出双策略框架,让机器人主动找信息再决策。
Learning to Look: Seeking Information for Decision Making via Policy Factorization

- 拆解决策与寻信为两个独立策略,分别训练。
- 真实场景中五类任务表现优于现有方法。
- 适合需要主动感知的机器人操作任务。
许多机器人操作任务需要主动或交互式探索才能成功完成,这类任务在具身智能领域普遍存在:代理必须主动搜索任务各阶段所需的信息,例如移动机器人的头部以获取与操作相关的信息,或在多机器人场景中,由侦察机器人寻找另一机器人决策所需的信息。本文将此类任务定义为一种新型问题——分解式上下文马尔可夫决策过程(factorized Contextual Markov Decision Processes),并提出DISaM,一种双策略解决方案:一个信息搜寻策略负责探索环境以获取相关上下文信息,一个信息接收策略则利用上下文实现操作目标。该分解允许两个策略独立训练,且通过信息接收策略提供奖励来指导信息搜寻策略的学习。测试时,双代理根据操作策略对下一步最佳动作的不确定性,动态平衡探索与利用。我们在五个需信息搜寻行为的操作任务中验证了该方法的有效性,涵盖仿真与真实世界,DISaM显著优于现有方法。
原文摘要 · Abstract (English)
Many robot manipulation tasks require active or interactive exploration behavior in order to be performed successfully. Such tasks are ubiquitous in embodied domains, where agents must actively search for the information necessary for each stage of a task, e.g., moving the head of the robot to find information relevant to manipulation, or in multi-robot domains, where one scout robot may search for the information that another robot needs to make informed decisions. We identify these tasks with a new type of problem, factorized Contextual Markov Decision Processes, and propose DISaM, a dual-policy solution composed of an information-seeking policy that explores the environment to find the relevant contextual information and an information-receiving policy that exploits the context to achieve the manipulation goal. This factorization allows us to train both policies separately, using the information-receiving one to provide reward to train the information-seeking policy. At test time, the dual agent balances exploration and exploitation based on the uncertainty the manipulation policy has on what the next best action is. We demonstrate the capabilities of our dual policy solution in five manipulation tasks that require information-seeking behaviors, both in simulation and in the real-world, where DISaM significantly outperforms existing methods. More information at https://robin-lab.cs.utexas.edu/learning2look/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。