用博弈论设计抗干扰的智能体策略，确保复杂任务在对手干扰下仍能完成。
STLGame: Signal Temporal Logic Games in Adversarial Multi-Agent Systems
- 将多智能体系统建模为对抗博弈，用纳什均衡保障策略鲁棒性。
- 在连续空间中通过可微分STL公式实现最优响应逼近，实验表明几乎无法被攻破。
- 适用于自动驾驶车辆与无人机等动态系统，对未知对手策略有强适应性。
我们研究如何在对抗环境下，针对信号时序逻辑（STL）任务合成鲁棒且安全的自主系统策略。为确保最坏情况下的STL满足性，提出STLGame框架，将多智能体系统建模为双人零和博弈：自车智能体最大化STL满足度，其他智能体最小化该值。STLGame旨在通过虚构自博弈（FSP）框架寻找纳什均衡策略组合，该组合在面对未见对手策略时表现最优。尽管游戏定义于连续状态-动作空间，FSP仍可迭代收敛至纳什解。为此，提出基于梯度的可微分STL方法，用于在每轮FSP中近似最佳响应。实验对比显示，该方法在阿克曼转向车辆与自主无人机两个标准动力学系统基准上，所得策略几乎不可被利用，对多种未知对手策略表现出显著鲁棒性。所有代码与补充结果详见项目网站：https://sites.google.com/view/stlgame。
原文摘要 · Abstract (English)
We study how to synthesize a robust and safe policy for autonomous systems under signal temporal logic (STL) tasks in adversarial settings against unknown dynamic agents. To ensure the worst-case STL satisfaction, we propose STLGame, a framework that models the multi-agent system as a two-player zero-sum game, where the ego agents try to maximize the STL satisfaction and other agents minimize it. STLGame aims to find a Nash equilibrium policy profile, which is the best case in terms of robustness against unseen opponent policies, by using the fictitious self-play (FSP) framework. FSP iteratively converges to a Nash profile, even in games set in continuous state-action spaces. We propose a gradient-based method with differentiable STL formulas, which is crucial in continuous settings to approximate the best responses at each iteration of FSP. We show this key aspect experimentally by comparing with reinforcement learning-based methods to find the best response. Experiments on two standard dynamical system benchmarks, Ackermann steering vehicles and autonomous drones, demonstrate that our converged policy is almost unexploitable and robust to various unseen opponents' policies. All code and additional experimental results can be found on our project website: https://sites.google.com/view/stlgame
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。