为科学机器学习代理设计可验证的无真值测试方法
Domain-Validity-Gated Metamorphic Testing of Scientific ML Surrogates
- 构建领域有效性评判标准,筛选真正有意义的测试关系
- 在多个流体仿真任务中验证了测试结果的稳定性和可复现性
- 适合需要可信验证的科学计算领域研究人员使用
科学机器学习(SciML)代理用于替代高成本模拟,但任意输入的精确输出无法获得(即真值问题)。变态测试通过检查多次执行间的关联关系进行验证,然而候选关系并非自动有效:其前提条件、输出映射及评分算子的数值下限共同决定异常是否具有意义。本文研究如何对候选变态关系(MRs)进行领域有效性筛选,并转化为可执行的无真值测试资产。提出:(i) 领域有效性准则,仅当容差超过算子数值下限且前提成立时才接受候选;(ii) MR卡可执行资产格式,记录源用例、变换、指标、容差与类型化关系级判定结果;(iii) MeshGraphNets圆柱流代理案例研究协议,通过声明账本将每项结果关联到可追踪的产物。在MeshGraphNets检查点上,节点置换达到机器精度,镜像对称性为有界分布外压力测试而非严格对称,绝对守恒保持未定,而参考相对守恒通过验证。相同结论在保留轨迹、检查点清单、三种其他架构及PhysicsNeMo中一致。在第二个CFD任务(可压缩机翼)中,谓词基于物理理由拒绝不可压缩连续性,表明其理解领域有效性而非固定清单。在另一类偏微分方程(FNO Burgers与热方程)中,生成完整接受/拒绝/执行判定。证据涵盖两个CFD任务和一类额外的偏微分方程家族,支持从候选关系到可审计测试资产的有效性感知桥梁,实现模型违规与域外应用的分离。
原文摘要 · Abstract (English)
Scientific machine-learning (SciML) surrogates approximate expensive simulations, but exact expected outputs for arbitrary inputs are unavailable (the oracle problem). Metamorphic testing checks relations across executions, yet a candidate relation is not automatically valid: its preconditions, output mapping, and the numerical floor of the scoring operator determine whether a violation is meaningful. We study how candidate metamorphic relations (MRs) can be screened for domain validity and turned into executable, oracle-free test assets for SciML surrogates. We propose (i) a domain-validity rubric that admits a candidate only when its tolerance dominates the operator's numerical floor and its preconditions hold; (ii) an MR-card executable-asset format recording source cases, transformations, metrics, tolerances, and typed relation-level verdicts; and (iii) a case-study protocol on MeshGraphNets cylinder-flow surrogates, with a claim ledger binding every result to a tracked artifact. On a MeshGraphNets checkpoint, node permutation holds to machine precision, mirror-y is a bounded out-of-distribution stress finding rather than an exact symmetry, and absolute conservation stays deferred while a reference-relative guard passes. The same readings hold across held-out trajectories, a checkpoint roster, three further architectures, and PhysicsNeMo. On a second CFD task (compressible airfoil) the predicate instead rejects incompressible continuity on physical grounds, showing it reasons about domain validity rather than running a fixed checklist. On a second PDE family, FNO Burgers and heat surrogates run full admit/reject/execute verdicts. The evidence spans two CFD tasks and a second PDE family, supporting a validity-aware bridge from candidate MRs to auditable SciML test assets that separates model-level violations from out-of-domain applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。