根据成本收益比自动停止测试，用少部分标注发现多数缺陷。
AdaStop: Cost-Aware Early Stopping for DNN Test Selection
- 动态评估发现缺陷的效率，低于成本收益阈值时停止标注。
- 仅用9%-31%的标注预算即可发现65%-84%的模型缺陷。
- 适合资源受限场景下的高效模型测试，尤其关注成本控制。
现有深度神经网络（DNN）测试方法通常在固定标注预算下优先选择可能暴露模型故障的测试输入。但在实际中，确定该预算十分困难：预算过低会遗漏故障，过高则带来不必要的标注成本。本文研究DNN测试中的停止问题，将测试建模为一个成本-收益决策过程——标注每个输入的成本为 $c$，发现故障的收益为 $v$。基于此，提出 extit{AdaStop} 框架，通过估计测试过程中边际故障发现率，在其低于阈值 $τ = c/v$ 时停止标注。在多个数据集、模型架构和选择策略上的实验表明，仅使用 9–31% 的标注预算，即可发现 65–84% 的故障。
原文摘要 · Abstract (English)
Existing methods for testing deep neural networks (DNNs) primarily prioritize test inputs likely to reveal model faults under a fixed labeling budget. In practice, choosing that budget is difficult: too little testing misses failures, while too much incurs unnecessary labeling costs. This work studies the stopping problem in DNN testing. We formulate testing as a cost--benefit decision process in which labeling an input incurs cost $c$ and discovering a fault yields value $v$. Based on this formulation, we introduce \textit{AdaStop}, a framework that estimates the marginal fault discovery rate during testing and stops labeling when the estimated rate falls below the threshold $τ= c/v$. Experiments across multiple datasets, architectures, and selection strategies show that $65$--$84\%$ of faults can be discovered using only $9$--$31\%$ of the labeling budget.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。