详解机器人学习中的深度强化与模仿学习基础方法
An Introduction to Deep Reinforcement and Imitation Learning
- 从马尔可夫决策过程出发，系统讲解DRL与DIL核心算法
- 涵盖PPO、GAIL等经典方法，突出策略优化与示范学习机制
- 适合想打牢强化学习基础的初学者快速入门
具身智能体（如机器人和虚拟角色）需持续选择动作以有效完成任务，解决复杂的序列决策问题。由于人工设计控制器难度大，基于学习的方法成为重要替代方案，尤其是深度强化学习（DRL）与深度模仿学习（DIL）。DRL利用奖励信号优化行为，DIL则通过专家示范引导学习。本文在具身智能体背景下介绍DRL与DIL，采用简洁、深度优先的文献梳理方式，内容自洽，随需引入必要的数学与机器学习概念。本文非领域综述，而是聚焦少数基础算法与技术，强调深入理解而非广度覆盖。内容涵盖马尔可夫决策过程至REINFORCE与近端策略优化（PPO）等DRL方法，以及行为克隆至数据集聚合（DAgger）与生成对抗模仿学习（GAIL）等DIL方法。
原文摘要 · Abstract (English)
Embodied agents, such as robots and virtual characters, must continuously select actions to execute tasks effectively, solving complex sequential decision-making problems. Given the difficulty of designing such controllers manually, learning-based approaches have emerged as promising alternatives, most notably Deep Reinforcement Learning (DRL) and Deep Imitation Learning (DIL). DRL leverages reward signals to optimize behavior, while DIL uses expert demonstrations to guide learning. This document introduces DRL and DIL in the context of embodied agents, adopting a concise, depth-first approach to the literature. It is self-contained, presenting all necessary mathematical and machine learning concepts as they are needed. It is not intended as a survey of the field; rather, it focuses on a small set of foundational algorithms and techniques, prioritizing in-depth understanding over broad coverage. The material ranges from Markov Decision Processes to REINFORCE and Proximal Policy Optimization (PPO) for DRL, and from Behavioral Cloning to Dataset Aggregation (DAgger) and Generative Adversarial Imitation Learning (GAIL) for DIL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。