用语义不变变换测试代码模型鲁棒性，发现关键挑战与方向
Metamorphic Testing of Deep Code Models: A Systematic Literature Review
- 通过变量重命名等变换生成测试用例，检验代码模型输出稳定性
- 分析45篇论文发现主流模型多用于代码补全与缺陷检测任务
- 适合关注AI辅助编程安全性的研究人员与工程师参考
大型语言模型和深度学习模型在代码智能领域引发变革，可高精度完成代码补全、缺陷检测和代码摘要等任务，有望成为现代软件工程的核心。然而，这些模型在变量重命名等对抗性条件下仍可能产生不一致结果，鲁棒性成关键问题。元测试（Metamorphic Testing）通过施加语义保持变换并分析输出稳定性，成为评估模型鲁棒性的常用方法。本文系统综述45篇相关论文，分析所用变换方式、技术手段与评估方法，总结当前研究热点：常见模型包括CodeBERT、GraphCodeBERT；主要任务为代码补全与缺陷检测；数据集涵盖Java、Python等；常用指标包括准确率与一致性率。研究指出，现有工作在变换设计与评估标准化方面仍存在不足，未来需加强跨语言泛化与自动化测试框架建设。
原文摘要 · Abstract (English)
Large language models and deep learning models designed for code intelligence have revolutionized the software engineering field due to their ability to perform various code-related tasks. These models can process source code and software artifacts with high accuracy in tasks such as code completion, defect detection, and code summarization; therefore, they can potentially become an integral part of modern software engineering practices. Despite these capabilities, robustness remains a critical quality attribute for deep-code models as they may produce different results under varied and adversarial conditions (e.g., variable renaming). Metamorphic testing has become a widely used approach to evaluate models' robustness by applying semantic-preserving transformations to input programs and analyzing the stability of model outputs. While prior research has explored testing deep learning models, this systematic literature review focuses specifically on metamorphic testing for deep code models. By studying 45 primary papers, we analyze the transformations, techniques, and evaluation methods used to assess robustness. Our review summarizes the current landscape, identifying frequently evaluated models, programming tasks, datasets, target languages, and evaluation metrics, and highlights key challenges and future directions for advancing the field.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。