让智能体模仿人类的‘够用就好’行为，更贴近真实演示者的真实意图。
Imitation Learning via Focused Satisficing
- 基于可接受性阈值设计新目标，不显式学习演示者的期望水平。
- 在未见演示上实现更高可接受率，同时保持与最优方法相当的真实回报。
- 适合处理非最优但可用的演示数据，尤其适用于新手或动态目标场景。
模仿学习通常假设示范行为接近某个固定但未知的成本函数下的最优解。然而，根据满意理论，人类常依据个人（且可能动态变化的）期望水平选择可接受的行为，而非追求近似最优。例如，一个着陆器示范若成功着陆而未坠毁，对初学者而言已足够可接受，即使过程缓慢或抖动明显。本文提出一种基于边距的目标函数的聚焦满意模仿学习方法，旨在生成超越示范者期望水平（定义在轨迹或其部分上）的策略，且无需显式学习这些期望。实验表明，该方法能更有效地模仿高质量（轨迹片段），在未见示范上获得更高的保证可接受率，并在多种环境上达到与现有方法相当的真实回报。
原文摘要 · Abstract (English)
Imitation learning often assumes that demonstrations are close to optimal according to some fixed, but unknown, cost function. However, according to satisficing theory, humans often choose acceptable behavior based on their personal (and potentially dynamic) levels of aspiration, rather than achieving (near-) optimality. For example, a lunar lander demonstration that successfully lands without crashing might be acceptable to a novice despite being slow or jerky. Using a margin-based objective to guide deep reinforcement learning, our focused satisficing approach to imitation learning seeks a policy that surpasses the demonstrator's aspiration levels -- defined over trajectories or portions of trajectories -- on unseen demonstrations without explicitly learning those aspirations. We show experimentally that this focuses the policy to imitate the highest quality (portions of) demonstrations better than existing imitation learning methods, providing much higher rates of guaranteed acceptability to the demonstrator, and competitive true returns on a range of environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。