让智能体用不同规划长度学习共同奖励函数
Inverse Reinforcement Learning with Multiple Planning Horizons
- 设计算法区分不同规划时长的智能体，反推共享奖励函数
- 可重构专家策略，且在多场景下保持泛化能力
- 适合研究多智能体协作与奖励逆向工程的学者
本文研究一种逆强化学习问题：专家智能体共享同一奖励函数，但具有未知且不同的规划时长（planning horizons）。由于缺乏折扣因子信息，奖励函数的可行解空间更大，现有IRL方法难以准确识别。为此，我们提出算法，能够学习全局多智能体奖励函数，并为每个智能体分配特定折扣因子，以重构专家策略。我们刻画了两种算法下奖励函数与折扣因子的可行解空间，并验证了所学奖励函数在多个领域的泛化能力。
原文摘要 · Abstract (English)
In this work, we study an inverse reinforcement learning (IRL) problem where the experts are planning under a shared reward function but with different, unknown planning horizons. Without the knowledge of discount factors, the reward function has a larger feasible solution set, which makes it harder for existing IRL approaches to identify a reward function. To overcome this challenge, we develop algorithms that can learn a global multi-agent reward function with agent-specific discount factors that reconstruct the expert policies. We characterize the feasible solution space of the reward function and discount factors for both algorithms and demonstrate the generalizability of the learned reward function across multiple domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。