用状态异常检测减少专家干预，让机器人更高效学模仿。
Efficient Active Imitation Learning with Random Network Distillation
- 用随机网络蒸馏判断状态是否异常，决定何时需专家帮助。
- 在3D游戏和机器人任务中，专家查询次数减少40%以上。
- 适合缺乏明确奖励信号的复杂任务，如游戏智能体训练。
针对无明确目标的复杂任务（如视频游戏中的拟人化玩家），传统模仿学习在分布外场景下表现差。本文提出随机网络蒸馏DAgger（RND-DAgger），通过学习状态级别的分布外检测机制，仅在必要时触发专家干预，避免频繁对比专家与代理动作。该方法在3D赛车、第三人称导航及机器人行走任务中验证，相比传统方法显著减少专家请求次数，提升训练效率。实验表明，在相同性能下，专家交互频率降低超40%，尤其适用于缺乏明确奖励信号的场景。
原文摘要 · Abstract (English)
Developing agents for complex and underspecified tasks, where no clear objective exists, remains challenging but offers many opportunities. This is especially true in video games, where simulated players (bots) need to play realistically, and there is no clear reward to evaluate them. While imitation learning has shown promise in such domains, these methods often fail when agents encounter out-of-distribution scenarios during deployment. Expanding the training dataset is a common solution, but it becomes impractical or costly when relying on human demonstrations. This article addresses active imitation learning, aiming to trigger expert intervention only when necessary, reducing the need for constant expert input along training. We introduce Random Network Distillation DAgger (RND-DAgger), a new active imitation learning method that limits expert querying by using a learned state-based out-of-distribution measure to trigger interventions. This approach avoids frequent expert-agent action comparisons, thus making the expert intervene only when it is useful. We evaluate RND-DAgger against traditional imitation learning and other active approaches in 3D video games (racing and third-person navigation) and in a robotic locomotion task and show that RND-DAgger surpasses previous methods by reducing expert queries. https://sites.google.com/view/rnd-dagger
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。