让机器人足球战术决策可解释,用语义状态捕捉攻防关键点
Expandable Decision-Making States for Multi-Agent Deep Reinforcement Learning in Soccer Tactical Analysis
- 用关系变量扩展位置速度数据,构建可解释的战术状态空间
- 结合动作掩码机制,区分控球与非控球球员决策路径
- 结果更符合比赛规则,可视化展现高风险高回报战术模式
足球等侵入性团队运动具有高维强耦合的状态空间,众多球员在共享场地上持续交互,给定量战术分析带来挑战。传统规则分析直观但缺乏泛化能力,现代机器学习模型常做模式匹配而无明确代理表征。本文提出可扩展决策状态(EDMS),在原始位置和速度基础上加入关系变量(如空间得分、传球与射门机会),并设计动作掩码机制,使持球与非持球球员拥有不同决策集合。相比以往方法,EDMS将学习到的价值函数与策略映射至人类可理解的战术概念(如逼抢压力、传球路线、控球可达性),且与比赛规则对齐。实验表明,引入动作掩码后,行动预测损失与时序差分误差均持续降低。定性案例与Q值可视化进一步揭示了快速反击、防守突破等高风险高回报战术模式。本方法已集成至开源库,兼容多个商业与公开数据集,支持跨平台评估与可复现研究。
原文摘要 · Abstract (English)
Invasion team sports such as soccer produce a high-dimensional, strongly coupled state space as many players continuously interact on a shared field, challenging quantitative tactical analysis. Traditional rule-based analyses are intuitive, while modern predictive machine learning models often perform pattern-matching without explicit agent representations. The problem we address is how to build player-level agent models from data, whose learned values and policies are both tactically interpretable and robust across heterogeneous data sources. Here, we propose Expandable Decision-Making States (EDMS), a semantically enriched state representation that augments raw positions and velocities with relational variables (e.g., scoring of space, pass, and score), combined with an action-masking scheme that gives on-ball and off-ball agents distinct decision sets. Compared to prior work, EDMS maps learned value functions and action policies to human-interpretable tactical concepts (e.g., marking pressure, passing lanes, ball accessibility) instead of raw coordinate features, and aligns agent choices with the rules of play. In the experiments, EDMS with action masking consistently reduced both action-prediction loss and temporal-difference (TD) error compared to the baseline. Qualitative case studies and Q-value visualizations further indicate that EDMS highlights high-risk, high-reward tactical patterns (e.g., fast counterattacks and defensive breakthroughs). We also integrated our approach into an open-source library and demonstrated compatibility with multiple commercial and open datasets, enabling cross-provider evaluation and reproducible experiments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。