AI生成的代码常缺安全防护,反复提问才能补全漏洞。
The Illusion of Secure LLM Code: Closing the Security Gap via Iterative Reprompting
- 用迭代追问让模型自我审查,提升代码安全性
- 单次提示下5个主流AI均漏掉关键防护机制
- 适合企业级开发团队构建可信AI辅助流程
大型语言模型(LLMs)被广泛应用于软件开发,但其自动生成安全认证代码的能力仍存疑。本文通过静态分析与动态渗透测试相结合的双模评估框架,依据NIST SP 800-63B标准,评估五款主流AI编程助手生成的认证系统。研究对比了基础提示、安全提示、NIST提示和迭代追问四种策略下的模型表现。结果表明,功能型或通用安全提示生成的代码普遍缺失关键防护,尤其在防暴力破解、会话管理及强密码处理方面。尽管提供明确的NIST上下文能显著提升合规性,但仍存在结构性缺陷。只有通过迭代追问,迫使模型进入上下文自我审计循环,才能实现全面的纵深防御架构。研究证实,当前AI编程助手无法默认生成安全代码,企业部署必须从单次提示转向持续、标准驱动的验证流程。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly integrated into software development workflows, yet their ability to autonomously generate secure authentication code remains uncertain. This paper evaluates the security architecture of authentication systems generated by five prominent AI coding assistants through a bi-modal assessment framework combining static code analysis and dynamic penetration testing, mapped to NIST SP 800-63B guidelines. The study examines model behavior across four prompting strategies Basic, Secure, NIST-Based, and Reprompting to reflect varying levels of developer guidance. Empirical results demonstrate that code generated from functional or generically secure prompts consistently omits critical protections, particularly concerning brute-force resistance, session management, and robust password handling. While providing explicit, single-shot NIST context significantly improves compliance, the findings reveal that this remains structurally inadequate. Instead, iterative Reprompting: forcing models into a contextual self-auditing loop is strictly required to achieve a comprehensive, defense-in-depth security architecture. Ultimately, this study proves that current AI coding assistants do not produce secure-by-default applications, dictating that enterprise deployments must transition from single-shot prompt engineering to continuous, standards-driven verification pipelines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。