让智能体仅通过观察专家状态来学习，突破动作信息难获取的限制。
Learning from Observation: A Survey of Recent Advances
- 只用专家的状态轨迹，不依赖动作信息进行模仿学习
- 提出系统分类框架，梳理现有方法的核心设计差异
- 关联离线强化学习等方向，指明未来研究突破口
模仿学习（IL）通过模仿专家行为训练智能体，无需奖励函数。传统方法需访问专家的状态与动作信息，但在现实场景中动作数据常难以获取。为此，近年兴起“从观察中学习”（LfO）或“仅状态模仿学习”（SOIL），即智能体仅能获取专家的状态访问信息。本文提出一个LfO的分析框架，用于系统梳理和分类现有方法，涵盖轨迹构建方式、假设条件及算法设计选择。同时，文章揭示了离线强化学习、基于模型的强化学习与分层强化学习等领域的关联。最后，基于该框架识别出关键开放问题，并提出未来研究方向。
原文摘要 · Abstract (English)
Imitation Learning (IL) algorithms offer an efficient way to train an agent by mimicking an expert's behavior without requiring a reward function. IL algorithms often necessitate access to state and action information from expert demonstrations. Although expert actions can provide detailed guidance, requiring such action information may prove impractical for real-world applications where expert actions are difficult to obtain. To address this limitation, the concept of learning from observation (LfO) or state-only imitation learning (SOIL) has recently gained attention, wherein the imitator only has access to expert state visitation information. In this paper, we present a framework for LfO and use it to survey and classify existing LfO methods in terms of their trajectory construction, assumptions and algorithm's design choices. This survey also draws connections between several related fields like offline RL, model-based RL and hierarchical RL. Finally, we use our framework to identify open problems and suggest future research directions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。