机器人无需奖励信号,自主生成可复用的子目标链以实现长期学习。
Autonomous Generation of Sub-goals for Lifelong Learning in Robots
- 通过自上而下与自下而上的双路径方法,从通用目标中衍生子目标。
- 在真实机器人上验证,能高效生成子目标并复用技能,减少重复。
- 适合需要长期自主学习的机器人系统,如服务机器人、探索型智能体。
机器人在开放式学习中面临自主发现目标并学习达成技能的挑战。在长期学习场景下,仅靠显式奖励难以持续推进,因此需不依赖外部奖励,自主生成子目标及其对应技能,作为达成目标的阶梯。本文提出一种双路径子目标生成方法:自上而下的方法基于内在动机,从一般目标分层推导子目标;自下而上的方法则通过显化先前在不同领域学习到的目标与感知类别的潜在关联,形成子目标链。该方法使机器人能自主生成并串联子目标,实现更通用目标。同时,构建了更高阶的目标表征,减少子目标重复,提升技能学习效率。该方法集成于现有终身开放式学习认知架构,并在真实机器人上测试,显著增强了机器人发现与达成目标的能力,支持高效子目标生成、技能泛化,以及在动态未知环境中无显式中间奖励条件下的运行。
原文摘要 · Abstract (English)
One of the challenges of open-ended learning in robots is the need to autonomously discover goals and learn skills to achieve them. However, when in lifelong learning settings, it is always desirable to generate sub-goals with their associated skills, without relying on explicit reward, as steppingstones to a goal. This allows sub-goals and skills to be reused to facilitate achieving other goals. This work proposes a two-pronged approach for sub-goal generation to address this challenge: a top-down approach, where sub-goals are hierarchically derived from general goals using intrinsic motivations to discover them, and a bottom-up approach, where sub-goal chains emerge from making latent relationships between goals and perceptual classes that were previously learned in different domains explicit. These methods help the robot to autonomously generate and chain sub-goals as a way to achieve more general goals. Additionally, they create more abstract representations of goals, helping to reduce sub-goal duplication and make the learning of skills more efficient. Implemented within an existing cognitive architecture for lifelong open-ended learning and tested with a real robot, our approach enhances the robot's ability to discover and achieve goals, generate sub-goals in an efficient manner, generalize learned skills, and operate in dynamic and unknown environments without explicit intermediate rewards.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。