开源高吞吐空战强化学习环境,支持多机型跨平台训练。
BVR Sim: An Open and High-Throughput Environment for Heterogeneous Air-Combat Reinforcement Learning

- 统一战术接口让策略跨异构战机使用
- C++后端每秒模拟104秒,10对10场景仍高效
- 仅用F-16数据训练的策略可适配4种新战机
超视距空战是具有部分可观测性、长时决策、能量管理与武器有限的强化学习难题。我们提出BVR Sim,一个开源的Gymnasium风格环境,专为异构空战强化学习设计。支持多种JSBSim飞机模型(如F-15、F-16、F/A-18、F-22),可配置武器、传感器、控制器与对手。统一战术动作接口在飞机特定内环控制器之上指定航向、高度、速度和开火指令,使策略能在异构平台间通用。环境提供可互换的Python与加速版C++后端,实体导向观测,组合奖励,脚本化对手,回放与可视化功能,并兼容多智能体学习框架。在0.4秒决策间隔下,C++后端在1对1场景中实现每秒104秒模拟时间,在10对10场景中仍具实用性。仅在F-16上训练的策略无需重训即可适配四种未见战机,经飞机特定控制器调整后达到45.5%平均胜率。MAPPO与HAPPO实验进一步验证其与标准多智能体强化学习流水线的端到端兼容性。
原文摘要 · Abstract (English)
Beyond-visual-range (BVR) air combat is a challenging reinforcement-learning domain characterized by partial observability, long-horizon decision making, energy management, and limited weapons. We present BVR Sim, an open-source Gymnasium-style environment designed for heterogeneous air-combat reinforcement learning. BVR Sim supports multiple JSBSim aircraft models, including the F-15, F-16, F/A-18, and F-22, with configurable weapons, sensors, controllers, and opponents. A unified tactical action interface specifies desired heading, altitude, speed, and weapon release above aircraft-specific inner-loop controllers, enabling policies to operate across heterogeneous platforms. The environment provides interchangeable Python and accelerated C++ backends, entity-oriented observations, compositional rewards, scripted opponents, replay and visualization, and adapters for multi-agent learning frameworks. At a 0.4-s decision interval, the C++ backend achieves 104 simulated seconds per wall-clock second in 1-vs-1 and remains practical through 10-vs-10 scenarios. A policy trained only on the F-16 transfers without retraining to four unseen aircraft, reaching a 45.5% mean win rate with aircraft-specific controller adaptation. MAPPO and HAPPO experiments further verify end-to-end compatibility with standard multi-agent reinforcement-learning pipelines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。