提出兼顾不确定性和学习效率的地震AI评估框架
Evaluation of Seismic Artificial Intelligence with Uncertainty
- 基于地震数据聚类构建三份数据集，分离训练随机性影响
- 通过多训练设计量化模型性能不确定性，揭示性能差异来源
- 帮助用户在不同数据量下选择最优模型并合理设定预期
人工智能已通过深度学习模型（DLMs）改变地震学领域，这些模型被训练用于完成特定任务。然而，目前仍缺乏稳健的评估框架来评价和比较DLMs。本文设计了一种评估框架，同时整合两个关键方面：性能不确定性与学习效率。为实现这一目标，我们采用针对地震数据定制的聚类方法，精心构建训练、验证和测试集，并实施大规模训练设计，以分离由随机训练过程和数据采样带来的性能不确定性。通过在三种训练策略下对流行相位拾取DLM PhaseNet [1] 的评估，展示了该框架防止误判模型优越性的能力。该框架使从业者能够在不同训练数据预算下，通过显式分析模型性能及其不确定性，选择最适合其问题的模型并设定合理的性能预期。
原文摘要 · Abstract (English)
Artificial intelligence has transformed the seismic community with deep learning models (DLMs) that are trained to complete specific tasks within workflows. However, there is still lack of robust evaluation frameworks for evaluating and comparing DLMs. We address this gap by designing an evaluation framework that jointly incorporates two crucial aspects: performance uncertainty and learning efficiency. To target these aspects, we meticulously construct the training, validation, and test splits using a clustering method tailored to seismic data and enact an expansive training design to segregate performance uncertainty arising from stochastic training processes and random data sampling. The framework's ability to guard against misleading declarations of model superiority is demonstrated through evaluation of PhaseNet [1], a popular seismic phase picking DLM, under 3 training approaches. Our framework helps practitioners choose the best model for their problem and set performance expectations by explicitly analyzing model performance with uncertainty at varying budgets of training data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。