BERTopic比PLSA更擅长从航空安全报告中提取有意义的主题。
Is BERTopic Better than PLSA for Extracting Key Topics in Aviation Safety Reports?
- 用Transformer嵌入+层次聚类，自动挖掘报告主题
- 主题连贯性得分0.41，优于PLSA的0.37
- 专家验证更易理解，适合安全分析人员使用
本研究比较了BERTopic与概率潜在语义分析（PLSA）在提取航空安全报告中关键主题的有效性，旨在深化对航空事故数据模式的理解。基于2000至2020年间超过36,000份美国国家运输安全委员会（NTSB）报告的数据集，BERTopic采用基于Transformer的嵌入和层次聚类，而PLSA则通过期望最大化（EM）算法进行概率建模。结果表明，BERTopic在主题连贯性上表现更优，其Cv得分为0.41，高于PLSA的0.37，同时在航空安全专家评估中展现出更好的可解释性。这些发现凸显了现代Transformer方法在分析复杂航空数据方面的优势，为提升航空安全洞察力和辅助决策提供了可能。未来工作将探索混合模型、多语言数据集及先进聚类技术以进一步优化该领域的话题建模。
原文摘要 · Abstract (English)
This study compares the effectiveness of BERTopic and Probabilistic Latent Semantic Analysis (PLSA) in extracting meaningful topics from aviation safety reports aiming to enhance the understanding of patterns in aviation incident data. Using a dataset of over 36,000 National Transportation Safety Board (NTSB) reports from 2000 to 2020, BERTopic employed transformer based embeddings and hierarchical clustering, while PLSA utilized probabilistic modelling through the Expectation-Maximization (EM) algorithm. Results showed that BERTopic outperformed PLSA in topic coherence, achieving a Cv score of 0.41 compared to PLSA 0.37, while also demonstrating superior interpretability as validated by aviation safety experts. These findings underscore the advantages of modern transformer based approaches in analyzing complex aviation datasets, paving the way for enhanced insights and informed decision-making in aviation safety. Future work will explore hybrid models, multilingual datasets, and advanced clustering techniques to further improve topic modelling in this domain.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。