提出系统级故障定位方法，精准识别深度学习系统全链路问题。
Fault Localization in Deep Learning-based Software: A System-level Approach
- 从整个深度学习开发流程出发，而非仅聚焦模型内部
- 在100个故障脚本上对6类故障定位准确率超前人方法
- 特别擅长发现数据、库版本不一致、损失函数等问题
过去十年，深度学习（DL）已深入日常应用。随着其广泛应用，构建可靠的深度学习软件系统变得愈发重要。故障定位是可靠性评估中的关键任务，现有研究多集中于深度学习模型内部的故障。然而，深度学习组件的性能不仅受模型影响，还受训练与部署环境等其他因素显著影响。因此，仅关注模型内部的故障定位方法会遗漏大量系统性问题。为此，本文提出FL4Deep，一种面向整个深度学习开发流程的系统级故障定位方法，可有效定位深度学习系统中的各类故障。在100个含故障的深度学习脚本上进行评估，结果表明，在六类深度学习相关故障中，FL4Deep在三类上的定位准确率超过四种现有方法：数据问题（84%）、训练与部署库版本不匹配（100%）、损失函数错误（69%）。此外，在五类故障（包含上述三类）的定位中，其精确率与召回率均表现更优，涵盖训练迭代不足和激活函数不当等问题。
原文摘要 · Abstract (English)
Over the past decade, Deep Learning (DL) has become an integral part of our daily lives. This surge in DL usage has heightened the need for developing reliable DL software systems. Given that fault localization is a critical task in reliability assessment, researchers have proposed several fault localization techniques for DL-based software, primarily focusing on faults within the DL model. While the DL model is central to DL components, there are other elements that significantly impact the performance of DL components. As a result, fault localization methods that concentrate solely on the DL model overlook a large portion of the system. To address this, we introduce FL4Deep, a system-level fault localization approach considering the entire DL development pipeline to effectively localize faults across the DL-based systems. In an evaluation using 100 faulty DL scripts, FL4Deep outperformed four previous approaches in terms of accuracy for three out of six DL-related faults, including issues related to data (84%), mismatched libraries between training and deployment (100%), and loss function (69%). Additionally, FL4Deep demonstrated superior precision and recall in fault localization for five categories of faults including three mentioned fault types in terms of accuracy, plus insufficient training iteration and activation function.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。