提出语义突变评分，更精准评估科学计算中的测试有效性。
A semantic mutation metric for metamorphic relation adequacy in scientific computing programs
- 基于五类领域语义操作构建新评分体系
- 实测显示语义突变效果属中等，非显著提升
- 适合关注科学计算测试质量的研究者
背景：元测试解决科学计算中的测试预言问题，但传统突变评分仅基于语法抽象语法树（AST）突变，忽略领域语义。目标：提出语义突变评分（SMS），基于五类领域语义操作（守恒退化、算子替换、超参数调整、轨迹翻转、结构注入）。在特定极限下，SMS 几乎处处退化为传统突变评分（MS），确保与既有文献一致性。方法：在四类单输出浮点到浮点程序（数值、概率、代理、机器学习）上，采用12个程序单元×5个突变策略设计，结合三层归因分类器区分真实语义故障与容错、域外、统计及人为干扰类别。在相同提示下进行同源/跨源消融实验，隔离大语言模型（LLM）来源差异影响。将LLM生成的突变体与默认配置的Cosmic-Ray语法突变池在AST归一化层面比较。结果：预注册的大效应阈值（Cliff's delta）未在点估计标准下达成，实际效应处于中等范围。相同提示下的跨源合并未显著改变delta值，表明该设计中LLM身份并非关键变量。在AST层面，LLM生成突变体与默认语法突变体重叠度低；超参数、结构注入和轨迹翻转类别在默认一阶语法配置下不可达。结论：SMS 是科学计算中领域语义元关系集的后向兼容性充分性度量。一阶不可达证据独立于效应大小问题。
原文摘要 · Abstract (English)
Context. Metamorphic Testing addresses the test-oracle problem in scientific computing, but classical Mutation Score operates on syntactic AST mutations and misses domain semantics. Objective. We propose the Semantic Mutation Score (SMS), built on five domain-semantic operators (Conservation Erosion, Operator Substitution, Hyperparameter, Trajectory Flip, Structural Injection). SMS degenerates almost everywhere to MS in a characterised limit, so any SMS-based conclusion remains consistent with prior mutation-testing literature in the classical regime. Method. A 12-PUT x 5-MP design over four single-output float-to-float classes (numeric, probabilistic, surrogate, machine-learning) is paired with a three-layer attribution classifier separating true semantic faults from tolerance, OOD, statistical, and artefact categories. A same-source / cross-source ablation under an identical prompt isolates the LLM-source-diversity contribution. LLM-generated mutants are compared against a default-configuration cosmic-ray syntactic pool at the AST-normalised level. Results. The pre-registered large-effect threshold for Cliff's delta is not met under the point-estimate criterion; the observed effect lies in the medium-effect range. Cross-source pooling under an identical prompt does not appreciably shift delta, indicating that LLM identity is not the lever within this design. AST-level overlap between LLM-generated and default cosmic-ray syntactic mutants is small; the Hyperparameter, Structural Injection, and Trajectory Flip classes are unreachable under default first-order syntactic configurations. Conclusion. SMS is a backward-compatible adequacy metric for domain-semantic metamorphic-relation sets in scientific computing. The first-order unreachability evidence is independent of the effect-size question.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。