用智能代理自动复现深度学习漏洞，成功率超80%
Imitation Game: Reproducing Deep Learning Bugs Leveraging an Intelligent Agent
- 构建项目上下文并用大模型迭代生成代码以复现漏洞
- 在106个真实漏洞上复现率达80.19%，提升近20个百分点
- 开发者测试显示效率提升56.8%，认知负担显著降低
尽管深度学习应用广泛分布于医疗、金融、软件工程等领域，但其仍存在诸多漏洞、故障与安全风险。复现这些漏洞对修复至关重要，但因深度学习模型的固有非确定性及其与软硬件环境的紧密耦合，复现极为困难。据近期研究，仅约3%的深度学习漏洞可通过人工方法可靠复现。为此，我们提出RepGen，一种新型自动化智能漏洞复现方法。RepGen从项目中构建学习增强型上下文，制定全面复现计划，采用迭代生成-验证-优化机制，利用大语言模型生成可复现特定漏洞的代码。在106个真实世界深度学习漏洞上评估，复现率达80.19%，较现有最佳方法提升19.81%。27名开发者的实证研究表明，RepGen使漏洞复现成功率提高23.35%，复现时间缩短56.8%，显著降低认知负荷。
原文摘要 · Abstract (English)
Despite their wide adoption in various domains (e.g., healthcare, finance, software engineering), Deep Learning (DL)-based applications suffer from many bugs, failures, and vulnerabilities. Reproducing these bugs is essential for their resolution, but it is extremely challenging due to the inherent nondeterminism of DL models and their tight coupling with hardware and software environments. According to recent studies, only about 3% of DL bugs can be reliably reproduced using manual approaches. To address these challenges, we present RepGen, a novel, automated, and intelligent approach for reproducing deep learning bugs. RepGen constructs a learning-enhanced context from a project, develops a comprehensive plan for bug reproduction, employs an iterative generate-validate-refine mechanism, and thus generates such code using an LLM that reproduces the bug at hand. We evaluate RepGen on 106 real-world deep learning bugs and achieve a reproduction rate of 80.19%, a 19.81% improvement over the state-of-the-art measure. A developer study involving 27 participants shows that RepGen improves the success rate of DL bug reproduction by 23.35%, reduces the time to reproduce by 56.8%, and lowers participants' cognitive load.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。