让智能体自动把经验提炼成可执行的高阶知识,持续进化。
Synthesizing Evolving Symbolic Representations for Autonomous Systems
- 用内在动机驱动探索,从零开始构建符号化知识表示
- 通过迭代发现选项、抽象经验、规划行动,实现知识闭环更新
- 适合研究自主系统长期学习与可解释决策的学者
近年来,人工智能在各类任务中取得显著进展。深度强化学习(DRL)是代理在低层状态空间中学习策略以解决复杂任务的有效工具。研究人员将内在动机(IM)引入强化学习机制,模拟代理的好奇心,鼓励其探索环境中的有趣区域。这一特性对代理在无明确目标情况下学习策略至关重要。然而,尽管DRL智能通过非符号模型涌现,仍需某种抽象来理解代理所积累的知识。为此,近期研究采用经典规划形式化显式表示自主代理获取的知识,并有效达成外在目标。虽然经典规划表达能力有限,但PPDDL已证明在回顾代理积累的知识、揭示显性因果关联方面具有实用性,并可用于生成到达代理经历任意状态的计划。本文提出一种新架构,实现一个开放式的学习系统,能够从零开始将经验合成到PPDDL表示中并随时间更新。在无预定义目标和任务的情况下,系统利用内在动机自主探索环境,利用经验中获得的高层知识。系统迭代地:(a) 发现选项,(b) 使用选项探索环境,(c) 抽象收集的知识,(d) 进行规划。该论文提出一种替代方案,通过结合低层与高层表示,实现知识的良性循环扩展。
原文摘要 · Abstract (English)
Recently, AI systems have made remarkable progress in various tasks. Deep Reinforcement Learning(DRL) is an effective tool for agents to learn policies in low-level state spaces to solve highly complex tasks. Researchers have introduced Intrinsic Motivation(IM) to the RL mechanism, which simulates the agent's curiosity, encouraging agents to explore interesting areas of the environment. This new feature has proved vital in enabling agents to learn policies without being given specific goals. However, even though DRL intelligence emerges through a sub-symbolic model, there is still a need for a sort of abstraction to understand the knowledge collected by the agent. To this end, the classical planning formalism has been used in recent research to explicitly represent the knowledge an autonomous agent acquires and effectively reach extrinsic goals. Despite classical planning usually presents limited expressive capabilities, PPDDL demonstrated usefulness in reviewing the knowledge gathered by an autonomous system, making explicit causal correlations, and can be exploited to find a plan to reach any state the agent faces during its experience. This work presents a new architecture implementing an open-ended learning system able to synthesize from scratch its experience into a PPDDL representation and update it over time. Without a predefined set of goals and tasks, the system integrates intrinsic motivations to explore the environment in a self-directed way, exploiting the high-level knowledge acquired during its experience. The system explores the environment and iteratively: (a) discover options, (b) explore the environment using options, (c) abstract the knowledge collected and (d) plan. This paper proposes an alternative approach to implementing open-ended learning architectures exploiting low-level and high-level representations to extend its knowledge in a virtuous loop.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。