arXiv:2502.06146cs.LGcs.AI2025-02

提出高效探索方法,让模型更快学会复杂环境中的关系规则。

Guided Exploration for Efficient Relational Model Learning

  • 用示范初始化算子,确保规划覆盖关键效果。
  • 通过选择有信息量的目标动作对,提升数据采集效率。
  • 适合需要高效学习关系模型的长程任务研究者。

在大规模、长时程的复杂环境中,高效探索对学习关系模型至关重要。随机探索常产生冗余或无关数据,限制模型准确性。目标字面乱动(GLIB)虽通过设定和规划新目标改进了随机探索,但其依赖随机动作和随机目标选择,难以扩展到更大领域。本文揭示了关系域中高效探索的核心原则:(1) 使用涵盖不同提升效应的示范初始化算子;(2) 通过选择有信息量的目标-动作对并执行计划,优化预条件以收集最具信息量的转移。为验证这些原则,我们引入了Baking-Large这一具有庞大状态-动作空间和长程任务的挑战性领域。实验采用基于先验知识的示范进行算子初始化,并使用预条件导向的引导来高效获取关键转移。结果表明,先验示范与预条件导向的引导均显著提升了样本效率与泛化能力,为未来方法在复杂领域高效学习准确关系模型提供了可行路径。

原文摘要 · Abstract (English)

Efficient exploration is critical for learning relational models in large-scale environments with complex, long-horizon tasks. Random exploration methods often collect redundant or irrelevant data, limiting their ability to learn accurate relational models of the environment. Goal-literal babbling (GLIB) improves upon random exploration by setting and planning to novel goals, but its reliance on random actions and random novel goal selection limits its scalability to larger domains. In this work, we identify the principles underlying efficient exploration in relational domains: (1) operator initialization with demonstrations that cover the distinct lifted effects necessary for planning and (2) refining preconditions to collect maximally informative transitions by selecting informative goal-action pairs and executing plans to them. To demonstrate these principles, we introduce Baking-Large, a challenging domain with extensive state-action spaces and long-horizon tasks. We evaluate methods using oracle-driven demonstrations for operator initialization and precondition-targeting guidance to efficiently gather critical transitions. Experiments show that both the oracle demonstrations and precondition-targeting oracle guidance significantly improve sample efficiency and generalization, paving the way for future methods to use these principles to efficiently learn accurate relational models in complex domains.

关系学习高效探索强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。