用大模型检测加密代码错误，效果比传统工具更好
Beyond Static Tools: Evaluating Large Language Models for Cryptographic Misuse Detection
- 对比静态分析工具与GPT、Gemini等大模型的检测能力
- GPT 4-o-mini在两个数据集上超越现有最佳工具
- 适合关注AI辅助安全编码的开发者和研究者
大型语言模型（LLMs）在软件开发中的应用迅速增长，开发者越来越多地依赖这些模型进行编码辅助，包括安全关键任务。本文系统比较了传统的加密API误用检测工具（CryptoGuard、CogniCrypt、Snyk Code）与LLMs（GPT和Gemini）的表现。基于OWASP、CryptoAPI和MASC三个基准数据集，评估各工具识别加密误用的有效性。结果显示，GPT 4-o-mini在CryptoAPI和MASC数据集上优于当前最先进静态分析工具，但在OWASP数据集上表现较差。同时，我们评估了大模型生成建议的质量，判断其是否具备可操作性和准确性，为开发者提供实际使用参考。本研究揭示了静态分析与大模型驱动方法的优劣，为人工智能在软件安全实践中的演进提供了重要洞察。
原文摘要 · Abstract (English)
The use of Large Language Models (LLMs) in software development is rapidly growing, with developers increasingly relying on these models for coding assistance, including security-critical tasks. Our work presents a comprehensive comparison between traditional static analysis tools for cryptographic API misuse detection-CryptoGuard, CogniCrypt, and Snyk Code-and the LLMs-GPT and Gemini. Using benchmark datasets (OWASP, CryptoAPI, and MASC), we evaluate the effectiveness of each tool in identifying cryptographic misuses. Our findings show that GPT 4-o-mini surpasses current state-of-the-art static analysis tools on the CryptoAPI and MASC datasets, though it lags on the OWASP dataset. Additionally, we assess the quality of LLM responses to determine which models provide actionable and accurate advice, giving developers insights into their practical utility for secure coding. This study highlights the comparative strengths and limitations of static analysis versus LLM-driven approaches, offering valuable insights into the evolving role of AI in advancing software security practices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。