arXiv:2503.13194cs.AIcs.LG2025-03

用多层次图结构增强智能体轨迹表示,提升复杂环境下的泛化能力。

A representational framework for learning and encoding structurally enriched trajectories in complex agent environments

  • 构建多层图结构的结构化轨迹(SET),融合物体、交互与功能关系
  • 在CREATE和MiniGrid环境中识别任务相关结构模式,支持下游任务
  • 结合强化学习后,在稀疏奖励任务中实现突破性成功率

人工智能代理在复杂场景中的最优决策与跨领域泛化能力受限。现有方法虽能高效学习世界表征,但缺乏结构性。本文提出结构化轨迹(SET),通过引入对象间的层级关系、交互与可及性,构建多层图结构,以更精细地刻画代理动态与任务功能抽象。将该机制集成至结构化轨迹学习与编码框架(SETLE),采用异构图记忆结构建模多层次关系依赖,显著提升泛化性能。实验表明,SETLE可在CREATE和MiniGrid环境中识别任务关键结构特征,并支持下游任务。进一步将其与强化学习结合,实现在复杂稀疏奖励任务中的显著性能提升,达成突破性成功率达87%。

原文摘要 · Abstract (English)

The ability of artificial intelligence agents to make optimal decisions and generalise them to different domains and tasks is compromised in complex scenarios. One way to address this issue has focused on learning efficient representations of the world and on how the actions of agents affect them in state-action transitions. Whereas such representations are procedurally efficient, they lack structural richness. To address this problem, we propose to enhance the agent's ontology and extend the traditional conceptualisation of trajectories to provide a more nuanced view of task execution. Structurally Enriched Trajectories (SETs) extend the encoding of sequences of states and their transitions by incorporating hierarchical relations between objects, interactions, and affordances. SETs are built as multi-level graphs, providing a detailed representation of the agent dynamics and a transferable functional abstraction of the task. SETs are integrated into an architecture, Structurally Enriched Trajectory Learning and Encoding (SETLE), that employs a heterogeneous graph-based memory structure of multi-level relational dependencies essential for generalisation. We demonstrate that SETLE can support downstream tasks, enabling agents to recognise task relevant structural patterns across CREATE and MiniGrid environments. Finally, we integrate SETLE with reinforcement learning and show measurable improvements in downstream performance, including breakthrough success rates in complex, sparse-reward tasks.

轨迹建模图神经网络强化学习泛化能力

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。