arXiv:2501.01690cs.LG2025-01被引 1

用三种模型分析百年航空安全文本,找出行之有效的风险识别方法。

Analyzing Aviation Safety Narratives with LDA, NMF and PLSA: A Case Study Using Socrata Datasets

  • 用LDA、NMF、PLSA三种主题模型挖掘航空事故文本中的隐藏主题。
  • NMF主题最清晰,LDA主题重叠性好,三者在不同指标上各有优劣。
  • 适合从事航空安全分析、文本挖掘的科研与工程人员参考。

本研究将隐含狄利克雷分布(LDA)、非负矩阵分解(NMF)和概率潜在语义分析(PLSA)应用于覆盖1908至2009年的Socrata航空安全数据集,按运营类型(军用、商用、私人)分类分析。识别出飞行员失误、机械故障、天气条件和训练不足等关键主题。统计结果显示:PLSA的主题一致性得分为0.32,困惑度为-4.6;NMF得分为0.34,困惑度为37.1;LDA一致性最高达0.36,但困惑度也最高为38.2。三者各具优势:LDA可发现主题重叠,NMF生成更清晰可解释的主题,PLSA提供精细概率推断但解释较复杂。研究证明主题模型能从非结构化航空安全文本中提取可行动洞察,助力识别风险因素与改进方向。未来可融合更多上下文变量、使用神经主题模型,并优化安全规程。该工作为航空安全管理中的高级文本挖掘奠定基础。

原文摘要 · Abstract (English)

This study explores the application of topic modelling techniques Latent Dirichlet Allocation (LDA), Nonnegative Matrix Factorization (NMF), and Probabilistic Latent Semantic Analysis (PLSA) on the Socrata dataset spanning from 1908 to 2009. Categorized by operator type (military, commercial, and private), the analysis identified key themes such as pilot error, mechanical failure, weather conditions, and training deficiencies. The study highlights the unique strengths of each method: LDA ability to uncover overlapping themes, NMF production of distinct and interpretable topics, and PLSA nuanced probabilistic insights despite interpretative complexity. Statistical analysis revealed that PLSA achieved a coherence score of 0.32 and a perplexity value of -4.6, NMF scored 0.34 and 37.1, while LDA achieved the highest coherence of 0.36 but recorded the highest perplexity at 38.2. These findings demonstrate the value of topic modelling in extracting actionable insights from unstructured aviation safety narratives, aiding in the identification of risk factors and areas for improvement across sectors. Future directions include integrating additional contextual variables, leveraging neural topic models, and enhancing aviation safety protocols. This research provides a foundation for advanced text-mining applications in aviation safety management.

主题模型航空安全文本挖掘

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。