arXiv:2511.18718cs.ROcs.AI2025-11

构建航空冲突检测的多人在线仿真平台,支持语音、视觉与雷达数据融合评估。

AIRHILT: A Human-in-the-Loop Testbed for Multimodal Conflict Detection in Aviation

  • 基于Godot引擎搭建多模态仿真环境,同步飞行员与空管通信、视觉与雷达数据。
  • 在跑道重叠场景中,系统平均7.7秒内发出首次预警,语音识别延迟约5.9秒。
  • 开源接口支持快速集成语音识别、目标检测等模型,适合航空安全研究者使用。

我们提出AIRHILT(航空综合推理、人机协同测试平台),一个模块化轻量级仿真环境,用于评估航空领域中飞行员与空中交通管制(ATC)辅助系统的多模态冲突检测能力。基于开源Godot引擎构建,AIRHILT在统一可扩展平台上同步飞行员与空管无线电通信、摄像头流的视觉理解以及ADS-B监视数据。该环境支持飞行员和管制员在环交互,提供覆盖终端区与航路运行冲突的完整场景集,包括通信错误与程序失误。通过标准化的JSON接口,研究人员可便捷集成、替换并评估自动语音识别(ASR)、视觉检测、决策与文本转语音(TTS)模型。我们以参考流程为例,采用微调后的Whisper ASR、基于YOLO的视觉检测、基于ADS-B的冲突逻辑及GPT-OSS-20B结构化推理,展示了典型跑道重叠场景的初步结果:系统平均首次预警时间为7.7秒,平均ASR与视觉延迟分别为5.9秒和0.4秒。AIRHILT环境与场景集已开源,支持可复现的航空多模态态势感知与冲突检测研究;代码与场景见https://github.com/ogarib3/airhilt。

原文摘要 · Abstract (English)

We introduce AIRHILT (Aviation Integrated Reasoning, Human-in-the-Loop Testbed), a modular and lightweight simulation environment designed to evaluate multimodal pilot and air traffic control (ATC) assistance systems for aviation conflict detection. Built on the open-source Godot engine, AIRHILT synchronizes pilot and ATC radio communications, visual scene understanding from camera streams, and ADS-B surveillance data within a unified, scalable platform. The environment supports pilot- and controller-in-the-loop interactions, providing a comprehensive scenario suite covering both terminal area and en route operational conflicts, including communication errors and procedural mistakes. AIRHILT offers standardized JSON-based interfaces that enable researchers to easily integrate, swap, and evaluate automatic speech recognition (ASR), visual detection, decision-making, and text-to-speech (TTS) models. We demonstrate AIRHILT through a reference pipeline incorporating fine-tuned Whisper ASR, YOLO-based visual detection, ADS-B-based conflict logic, and GPT-OSS-20B structured reasoning, and present preliminary results from representative runway-overlap scenarios, where the assistant achieves an average time-to-first-warning of approximately 7.7 s, with average ASR and vision latencies of approximately 5.9 s and 0.4 s, respectively. The AIRHILT environment and scenario suite are openly available, supporting reproducible research on multimodal situational awareness and conflict detection in aviation; code and scenarios are available at https://github.com/ogarib3/airhilt.

航空安全多模态仿真平台

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。