AI代码审查工具对严重安全漏洞识别率低，主要只改格式错误。
GitHub's Copilot Code Review: Can AI Spot Security Flaws Before You Commit?
- 用多语言开源项目样本测试GitHub Copilot代码审查功能
- 对SQL注入、XSS等关键漏洞识别率不足，常误判为低级错误
- 适合关注AI辅助开发局限性的安全工程师和研发管理者
随着软件开发日益依赖AI工具，确保其支持安全编码变得至关重要。本研究评估了GitHub Copilot新推出的代码审查功能在发现安全漏洞方面的表现。我们使用来自多个编程语言和应用领域的开源项目中精心筛选的已标注漏洞样本，系统性地测试了Copilot识别常见安全缺陷的能力。结果表明，与预期相反，Copilot在检测如SQL注入、跨站脚本（XSS）和不安全反序列化等关键漏洞时频繁失效，其反馈主要集中于低严重性问题，如代码风格和拼写错误。这一发现揭示了AI辅助代码审查的实际能力与其预期之间存在显著差距。研究强调，在保障软件安全方面，仍需依赖专用安全工具和人工代码审计。
原文摘要 · Abstract (English)
As software development practices increasingly adopt AI-powered tools, ensuring that such tools can support secure coding has become critical. This study evaluates the effectiveness of GitHub Copilot's recently introduced code review feature in detecting security vulnerabilities. Using a curated set of labeled vulnerable code samples drawn from diverse open-source projects spanning multiple programming languages and application domains, we systematically assessed Copilot's ability to identify and provide feedback on common security flaws. Contrary to expectations, our results reveal that Copilot's code review frequently fails to detect critical vulnerabilities such as SQL injection, cross-site scripting (XSS), and insecure deserialization. Instead, its feedback primarily addresses low-severity issues, such as coding style and typographical errors. These findings expose a significant gap between the perceived capabilities of AI-assisted code review and its actual effectiveness in supporting secure development practices. Our results highlight the continued necessity of dedicated security tools and manual code audits to ensure robust software security.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。