用形式化验证确保机器学习模型在地质灾害预测中符合物理规律。
Formal verification of tree-based machine learning models for lateral spreading
- 将树模型转为逻辑公式,用SMT求解器验证全输入域的物理一致性。
- 80.1%准确率的EBM违反所有四项地质规范,约束后仅三类满足。
- 揭示精度与物理一致性的固有权衡,适合安全关键型工程应用。
用于地质灾害预测的机器学习模型虽能实现高精度,但可能从稀疏或偏差数据中学习到物理不一致的关系。现有方法(如SHAP、LIME等事后解释或训练约束)或近似诊断个别预测,或限制模型能力而无法提供完整保证。本文将训练好的树集成模型编码为可满足性模理论(SMT)求解器中的逻辑公式,在整个输入空间而非采样点上验证物理规范。四个地质规范(地下水位深度、峰值加速度单调性、距离安全、平坦地面安全)被形式化为可判定逻辑公式,并通过SMT对基于2011年基督城地震横向滑移数据集(7,291个站点,4个特征)训练的XGBoost和可解释提升机(EBM)进行验证。SMT求解器可生成反例或证明无违规。未约束的EBM(准确率80.1%)违反全部四项规范;完全约束的EBM(准确率67.2%)满足其中三项,表明基于验证的迭代约束可逐步提升物理一致性。对33种模型变体的帕累托分析显示,不存在同时达到80%以上准确率且完全合规的模型。对反例的SHAP分析发现,违规特征排名可能靠后,说明事后解释无法替代形式化验证。研究建立了‘验证-修正-再验证’工程闭环,为安全关键地质应用提供形式化认证。
原文摘要 · Abstract (English)
Machine learning models for geotechnical hazard prediction can achieve high accuracy while learning physically inconsistent relationships from sparse or biased training data. Current remedies (post-hoc explainability, such as SHAP and LIME, and training-time constraints) either diagnose individual predictions approximately or restrict model capacity without providing exhaustive guarantees. This paper encodes trained tree ensembles as logical formulas in a Satisfiability Modulo Theories (SMT) solver and checks physical specifications across the entire input domain, not just sampled points. Four geotechnical specifications (water table depth, PGA monotonicity, distance safety, and flat-ground safety) are formalized as decidable logical formulas and verified via SMT against both XGBoost ensembles and Explainable Boosting Machines (EBMs) trained on the 2011 Christchurch earthquake lateral spreading dataset (7,291 sites, four features). The SMT solver either produces a concrete counterexample where a specification fails or proves that no violation exists. The unconstrained EBM (80.1% accuracy) violates all four specifications. A fully constrained EBM (67.2%) satisfies three of four specifications, demonstrating that iterative constraint application guided by verification can progressively improve physical consistency. A Pareto analysis of 33 model variants reveals a persistent trade-off, as none of the variants studied achieve both greater than 80% accuracy and full compliance with the specified set. SHAP analysis of specification counterexamples shows that the offending feature can rank last, demonstrating that post-hoc explanations do not substitute for formal verification. These results establish a verify-fix-verify engineering loop and a formal certification for deploying physically consistent ML models in safety-critical geotechnical applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。