arXiv:2605.26998cs.LGq-bio.NC2026-05

用递归网络建模行为中的目标切换,让机器更像人一样灵活变目标。

Probabilistic Recurrent Intention Switching Model

论文配图:Probabilistic Recurrent Intention Switching Model
图 1 · 摘自论文原文
  • 用轻量递归网络根据历史观察预测每一步的目标分布
  • 在三个场景中对未标注演示的预测准确率最高,且能提取可命名的目标序列
  • 适合研究人类/机器人多目标行为、需自动发现隐含目标的任务

逆强化学习(IRL)从观测行为中恢复奖励函数,但传统方法假设奖励恒定,无法捕捉单个轨迹内的目标切换。现有方法通过分段轨迹处理多意图,但意图转换要么是无记忆马尔可夫链,要么依赖固定历史窗口的人工状态增强。本文提出概率递归意图切换模型(PRISM),以轻量级递归网络替代上述机制,将观察历史映射为每步意图分布。我们证明该模型的期望最大化(EM)目标可精确分解为独立的每意图奖励子问题,每个均可闭式求解,实现 $/mathcal{O}(nK)$ 复杂度的E步且无需变分近似。在非马尔可夫网格世界、小鼠迷宫和BridgeData~V2机器人操作任务上评估,其中首次大规模应用多意图IRL于机器人操控。所有场景下PRISM均取得最高保留对数似然,且能从无标签演示中恢复出可命名、时间连贯的意图序列,表明生物与人工代理中均存在离散目标切换现象。

原文摘要 · Abstract (English)

Inverse reinforcement learning (IRL) recovers reward functions from observed behavior, yet traditional methods assume a single stationary reward that cannot capture goal switching within an episode. Recent multi-intention IRL methods address this by segmenting trajectories, but model intention transitions as either a memoryless Markov chain or via manual state augmentation with a fixed history window. We propose the Probabilistic Recurrent Intention Switching Model (PRISM), which replaces both mechanisms with a lightweight recurrent network that maps observation history to a per-step intention distribution. We prove that the resulting EM objective decomposes exactly into independent per-intention reward subproblems, each solvable in closed form, yielding an $\mathcal{O}(nK)$ E-step with no variational approximation. We evaluate PRISM on a non-Markovian gridworld, a mouse labyrinth, and BridgeData~V2 robotic manipulation, the first large-scale robotic application of multi-intention IRL. Across all settings PRISM achieves the highest held-out log-likelihood while recovering nameable, temporally coherent intentions from unlabeled demonstrations, suggesting that discrete goal switching is present in both biological and artificial agents.

逆强化学习意图建模递归网络多目标行为

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。