提出量化行为数据缺失程度的方法，让专家决策更真实可信。
Quantifying Potential Observation Missingness in Inverse Reinforcement Learning
- 通过最小扰动重构观测缺失，使专家行为在假设下显得最优
- 实验验证在导航、癌症治疗和ICU数据中可有效估计缺失程度
- 适合医疗等高风险决策场景的AI模型可靠性评估
逆强化学习（IRL）通过示范数据推断奖励函数，是理解人类决策行为的重要工具。尽管已有多种变体用于捕捉主观信念、规划不完美和动态目标等复杂性，但现实行为数据常存在决策者本可观察却未记录的缺失信息。在医疗等应用场景中，这会使原本近似最优的专家行为被误判为低效，导致标准IRL学习到的奖励函数产生误导。本文识别出使专家行为在给定信息下显得最优所需的最小观测缺失扰动，提出一种实用算法，并在合成导航任务、癌症治疗模拟器及ICU治疗数据上进行大量实验，成功量化了行为数据中可能存在的观测缺失范围。
原文摘要 · Abstract (English)
Inverse reinforcement learning (IRL), which infers reward functions from demonstrations, is a valuable tool for modeling and understanding decision-making behavior. Many variants of IRL have been developed to capture complexities of human decision-making, such as subjective beliefs, imperfect planning, and dynamic goals. However, an often-overlooked issue in real-world behavioral datasets is that the recorded data may be missing observations that were available to the original decision-maker. In use-inspired settings such as healthcare, this can make expert actions appear suboptimal, even when they were near-optimal given the information available at the time. As a result, the rewards learned by standard IRL may be misleading. In this paper, we identify the minimal perturbations to the recorded observations needed for the expert's actions to appear optimal. We develop a practical algorithm for this problem and demonstrate its utility for quantifying the possible extent of missing observations in behavioral datasets through extensive experiments on synthetic navigation tasks, a cancer treatment simulator, and ICU treatment data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。