用实时大数据框架构建空管系统，提升飞行数据处理效率
A Modern Approach to Real-Time Air Traffic Management System
- 采用Spark Streaming+Kafka实现飞行数据实时流处理
- 构建从API到Kibana的端到端可视化分析管道
- 适合交通管理、航空调度等实时决策场景
空中交通分析系统对保障航空安全、效率和可预测性至关重要。然而，传统系统难以应对日益增长和复杂的空中交通数据。本项目探索了Apache Spark、HDFS和Spark Streaming等实时大数据处理框架在构建新型稳健系统中的应用。通过回顾现有实时系统研究并分析大数据技术带来的挑战与机遇，我们提出了一种实时系统架构。项目流程包括通过航班API实时收集飞行信息，经Kafka接入，传输至Elasticsearch进行可视化，最终通过Kibana展示。此外，还展示了美国航空公司数据的PowerBI仪表盘，凸显实时分析在革新空中交通管理中的潜力。
原文摘要 · Abstract (English)
Air traffic analytics systems are pivotal for ensuring safety, efficiency, and predictability in air travel. However, traditional systems struggle to handle the increasing volume and complexity of air traffic data. This project explores the application of real-time big data processing frameworks like Apache Spark, HDFS, and Spark Streaming for developing a new robust system. By reviewing existing research on real-time systems and analyzing the challenges and opportunities presented by big data technologies, we propose an architecture for a real-time system. Our project pipeline involves real-time data collection from flight information sources through flight API's, ingestion into Kafka, and transmission to Elasticsearch for visualization using Kibana. Additionally, we present a dashboard of U.S. airlines on PowerBI, demonstrating the potential of real-time analytics in revolutionizing air traffic management.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。