用最大熵建模对手行为,让智能体更聪明地应对未知敌手。
TAB-Fields: A Maximum Entropy Framework for Mission-Aware Adversarial Planning
- 基于任务和环境约束,构建无偏的对手行为分布
- 在水下与地面机器人上验证,性能优于传统方法
- 适合对抗场景中需预判对手策略的规划任务
自主智能体在对抗环境中面临根本挑战:虽知对手的高层次目标(如在时限内到达特定位置),但其具体策略未知。传统方法将对手状态视为部分可观测,建模为部分可观测马尔可夫决策过程(POMDP),但其信念空间动态依赖于未知策略。本文提出任务感知行为场(TAB-Fields),通过求解约束优化问题,计算在已知任务与环境约束下最无偏的概率分布,刻画对手状态随时间的变化。我们设计了条件化POMCP算法(TAB-conditioned POMCP)将该表示集成至标准规划框架。仿真测试使用水下机器人,硬件实验基于地面机器人,结果表明,本方法在性能上显著优于假设特定对手策略或忽略任务约束的基线。相关视频与代码见https://tab-fields.github.io。
原文摘要 · Abstract (English)
Autonomous agents operating in adversarial scenarios face a fundamental challenge: while they may know their adversaries' high-level objectives, such as reaching specific destinations within time constraints, the exact policies these adversaries will employ remain unknown. Traditional approaches address this challenge by treating the adversary's state as a partially observable element, leading to a formulation as a Partially Observable Markov Decision Process (POMDP). However, the induced belief-space dynamics in a POMDP require knowledge of the system's transition dynamics, which, in this case, depend on the adversary's unknown policy. Our key observation is that while an adversary's exact policy is unknown, their behavior is necessarily constrained by their mission objectives and the physical environment, allowing us to characterize the space of possible behaviors without assuming specific policies. In this paper, we develop Task-Aware Behavior Fields (TAB-Fields), a representation that captures adversary state distributions over time by computing the most unbiased probability distribution consistent with known constraints. We construct TAB-Fields by solving a constrained optimization problem that minimizes additional assumptions about adversary behavior beyond mission and environmental requirements. We integrate TAB-Fields with standard planning algorithms by introducing TAB-conditioned POMCP, an adaptation of Partially Observable Monte Carlo Planning. Through experiments in simulation with underwater robots and hardware implementations with ground robots, we demonstrate that our approach achieves superior performance compared to baselines that either assume specific adversary policies or neglect mission constraints altogether. Evaluation videos and code are available at https://tab-fields.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。