分析多语言大模型生成代码的漏洞，发现安全能力随语言差异大。
Security and Quality in LLM-Generated Code: A Multi-Language, Multi-Model Analysis
- 构建200个任务数据集，覆盖六类编程场景测试
- 多数模型未使用新版编译器安全特性，如Java 17新功能
- 适合关注AI编码安全的开发者与工具链设计者
人工智能驱动的代码生成工具正广泛应用于软件开发全流程以加速编码。然而，大型语言模型（LLMs）生成代码的安全性仍缺乏充分研究，已有工作揭示出多种风险与缺陷。本文跨多编程语言分析了LLM生成代码的安全性。我们构建了一个包含200个任务、分为六类的评估数据集，用于衡量不同模型在生成安全且可维护代码方面的能力。研究发现，尽管LLMs能自动化代码生成，但其安全表现因编程语言而异。许多模型未能利用近年来编译器与开发工具包更新中的现代安全特性，例如Java 17的新功能。此外，尤其在C++中仍普遍使用过时的编程方法。这表明亟需推动LLMs演进，以提升代码安全性与质量，并融入编程语言领域的新兴最佳实践。
原文摘要 · Abstract (English)
Artificial Intelligence (AI)-driven code generation tools are increasingly used throughout the software development lifecycle to accelerate coding tasks. However, the security of AI-generated code using Large Language Models (LLMs) remains underexplored, with studies revealing various risks and weaknesses. This paper analyzes the security of code generated by LLMs across different programming languages. We introduce a dataset of 200 tasks grouped into six categories to evaluate the performance of LLMs in generating secure and maintainable code. Our research shows that while LLMs can automate code creation, their security effectiveness varies by language. Many models fail to utilize modern security features in recent compiler and toolkit updates, such as Java 17. Moreover, outdated methods are still commonly used, particularly in C++. This highlights the need for advancing LLMs to enhance security and quality while incorporating emerging best practices in programming languages.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。