让新手主动表达不确定,用计划信息减少人类教学次数。
ASkDAgger: Active Skill-level Data Aggregation for Interactive Imitation Learning
- 新手可主动报告计划与不确定性,教师仅在关键处反馈。
- 减少40%以上示范标注量,同时降低失败率并提升泛化能力。
- 适合需要高效人机协作的机器人学习场景。
交互式模仿学习中,人工教学成本高是主要瓶颈。现有方法虽通过主动学习仅在不确定或危险情境下提问,但忽略了新手计划中蕴含的能力与不确定性信息。为此,本文提出主动技能级数据聚合(ASkDAgger)框架,允许新手表达“我计划这么做,但不确定”。该框架通过三方面利用教师对新手计划的反馈:(1) 敏感性感知门控(SAG),动态调整决策阈值以匹配特定性能目标;(2) 前瞻式交互经验回放(FIER),将有效且重标注的新手计划转为示范数据;(3) 优先级交互经验回放(PIER),按不确定性、成功度和示范时效优先回放。实验在仿真与真实世界语言控制操作任务中验证了其有效性,显著减少示范标注需求,提升适应速度与泛化能力。代码、数据及视频见https://askdagger.github.io。
原文摘要 · Abstract (English)
Human teaching effort is a significant bottleneck for the broader applicability of interactive imitation learning. To reduce the number of required queries, existing methods employ active learning to query the human teacher only in uncertain, risky, or novel situations. However, during these queries, the novice's planned actions are not utilized despite containing valuable information, such as the novice's capabilities, as well as corresponding uncertainty levels. To this end, we allow the novice to say: "I plan to do this, but I am uncertain." We introduce the Active Skill-level Data Aggregation (ASkDAgger) framework, which leverages teacher feedback on the novice plan in three key ways: (1) S-Aware Gating (SAG): Adjusts the gating threshold to track sensitivity, specificity, or a minimum success rate; (2) Foresight Interactive Experience Replay (FIER), which recasts valid and relabeled novice action plans into demonstrations; and (3) Prioritized Interactive Experience Replay (PIER), which prioritizes replay based on uncertainty, novice success, and demonstration age. Together, these components balance query frequency with failure incidence, reduce the number of required demonstration annotations, improve generalization, and speed up adaptation to changing domains. We validate the effectiveness of ASkDAgger through language-conditioned manipulation tasks in both simulation and real-world environments. Code, data, and videos are available at https://askdagger.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。