测试大模型生成加密代码的漏洞，发现安全提示无法彻底消除风险
CIPHER: Cryptographic Insecurity Profiling via Hybrid Evaluation of Responses
- 用不安全/中性/安全三种提示对比生成代码漏洞
- 多数模型即使使用安全提示仍存可被利用的加密缺陷
- 适合关注代码安全的开发者和安全评估研究者
大型语言模型（LLMs）在辅助开发者编写代码方面应用日益广泛，但其生成的加密功能常包含可被利用的漏洞。微小的设计选择（如使用静态初始化向量或缺少认证）可能无声地破坏安全保证。我们提出CIPHER（Cryptographic Insecurity Profiling via Hybrid Evaluation of Responses），一个在受控安全引导条件下衡量LLM生成Python代码中加密漏洞发生率的基准。CIPHER针对每个任务采用不安全、中性和安全三种提示变体，结合特定于密码学的漏洞分类体系，并通过自动化评分流程实现行级归因。在多种广泛应用的LLM上测试发现，明确的安全提示虽能缓解部分问题，但无法可靠消除整体加密漏洞。该基准与可复现的评分流程将在发表后公开。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly used to assist developers with code, yet their implementations of cryptographic functionality often contain exploitable flaws. Minor design choices (e.g., static initialization vectors or missing authentication) can silently invalidate security guarantees. We introduce CIPHER(Cryptographic Insecurity Profiling via Hybrid Evaluation of Responses), a benchmark for measuring cryptographic vulnerability incidence in LLM-generated Python code under controlled security-guidance conditions. CIPHER uses insecure/neutral/secure prompt variants per task, a cryptography-specific vulnerability taxonomy, and line-level attribution via an automated scoring pipeline. Across a diverse set of widely used LLMs, we find that explicit secure prompting reduces some targeted issues but does not reliably eliminate cryptographic vulnerabilities overall. The benchmark and reproducible scoring pipeline will be publicly released upon publication.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。