arXiv:2504.02737cs.SEcs.AI2025-04被引 4

用自然语言需求生成测试用例,精准检测神经网络缺陷。

RBT4DNN: Requirements-based Testing of Neural Networks

  • 基于自然语言需求构建语义特征空间,生成满足前提条件的测试输入。
  • 生成的测试用例与需求前提高度一致,能有效发现模型缺陷。
  • 适合开发阶段验证模型泛化能力,提升测试针对性。

测试有助于判断系统是否按预期运行。当系统包含深度神经网络(DNN)时,测试变得困难,因为难以形式化其功能需求。为此,我们提出RBT4DNN方法,利用自然语言需求语句,通过术语词典定义语义特征空间,用于测试输入生成。RBT4DNN将功能需求的前置条件形式化为语义特征的逻辑组合,匹配这些特征的数据可微调生成模型,以可靠产生满足前提的测试输入。在训练好的DNN上执行这些测试,可对比输出与期望后置行为。该方法支持两个场景:(1) 给定定义DNN正确性属性的需求,可检测故障;(2) 开发过程中,需求引导的行为探索可提供模型泛化反馈。进一步评估表明,RBT4DNN生成的测试用例真实、多样且与需求前提对齐,支持针对模型行为的分析和高效故障检测。

原文摘要 · Abstract (English)

Testing allows developers to determine whether a system functions as expected. When such systems include deep neural networks (DNNs), Testing becomes challenging, as DNNs approximate functions for which the formalization of functional requirements is intractable. This prevents the application of well-developed approaches to requirements-based testing to DNNs. To address this, we propose a requirements-based testing method (RBT4DNN) that uses natural language requirements statements. These statements use a glossary of terms to define a semantic feature space that can be leveraged for test input generation. RBT4DNN formalizes preconditions of functional requirements as logical combinations of those semantic features. Training data matching these feature combinations can be used to fine-tune a generative model to reliably produce test inputs satisfying the precondition. Executing these tests on a trained DNN enables comparing its output to the expected requirement postcondition behavior. We propose two use cases for RBT4DNN: (1) given requirements defining DNN correctness properties, RBT4DNN comprises a novel approach for detecting faults, and (2) during development, requirements-guided exploration of model behavior can provide developers with feedback on model generalization. Our further evaluation shows that RBT4DNN-generated tests are realistic, diverse, and aligned with requirement preconditions, enabling targeted analysis of model behavior and effective fault detection.

神经网络测试需求驱动生成测试

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。