提出双维度评估框架,高效筛选深度学习中优质突变体。
Quality-Driven Selective Mutation for Deep Learning

- 基于抗杀性与真实感双指标量化突变体质量
- 可减少55.6%突变体生成量且保持效果
- 适合提升深度学习测试效率的开发者
突变体在深度学习测试与调试中扮演双重角色:作为测试目标或模拟真实故障的替代品。难以杀死的突变体能更好指导测试优化,而真实性对模拟真实缺陷至关重要。现有研究缺乏统一衡量标准。本文提出一种概率框架,从抗杀性与真实性两个互补维度量化突变体质量:抗杀性通过统计杀死概率扩展经典定义;真实性通过突变体与真实故障可检测性模式间的广义杰卡德相似度衡量。该框架支持无特定使用场景假设下的低质量配置筛选。在四个真实深度学习故障数据集上验证:三个(CleanML、DeepFD、DeepLocalize)用于估计与选择高质量操作符配置,另一保留数据集defect4ML用于验证。结果表明,质量驱动的选择可将突变体生成量降低至多55.6%,同时在基线对齐的筛选阈值下维持典型水平的抗杀性与真实性。证实双目标选择可在不牺牲突变体用途的前提下显著降低成本。
原文摘要 · Abstract (English)
Mutants support testing and debugging in two roles: (i) as test goals and (ii) as substitutes for real faults. Hard-to-kill mutants provide better guidance for test improvement, while realism is essential when mutants are used to simulate real bugs. Building on these roles, selective mutation for deep learning (DL) aims to reduce the cost of mutant generation and execution by choosing operator configurations that yield resistant and realistic mutants. However, the DL literature lacks a unified measure that captures both aspects. This study presents a probabilistic framework to quantify mutant quality along two complementary axes: resistance and realism. Resistance adapts the classical notion of hard-to-kill mutants to the DL setting using statistical killing probabilities, while realism is measured via the generalized Jaccard similarity between mutant and real-fault detectability patterns. The framework enables ranking and filtering of low-quality mutation-operator configurations without assuming a specific use case. We empirically evaluate the approach on four datasets of real DL faults. Three datasets (CleanML, DeepFD, and DeepLocalize) are used to estimate and select high-quality operator configurations, and the held-out defect4ML dataset is used for validation. Results show that quality-driven selection reduces the number of generated mutants by up to 55.6% while preserving typical levels of resistance and realism under baseline-aligned selection thresholds. These findings confirm that dual-objective selection can lower cost without compromising the usefulness of mutants for either role.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。