微小提示词改动可致代码漏洞，模型隐藏状态揭示脆弱性信号。
Minimal Prompt Perturbations Lead to Code Vulnerabilities: Prompt Fragility and Hidden-State Signals in Coding LLMs
- 通过字符级扰动测试，发现单字符变化即可引发安全漏洞。
- 输入处理类漏洞预测准确率均值达0.753，优于默认不安全类漏洞。
- 适合关注AI生成代码安全的开发者与安全研究员阅读。
基于大语言模型的编程助手正被广泛采用，显著提升开发效率。随着组织越来越多地部署这些代理生成的代码，其安全性变得至关重要。已有研究显示微小提示词扰动会降低生成代码的功能正确性，但其对代码安全的影响尚无研究。本文在三个模型和五种编程语言上施加逐标记扰动，发现仅一个字符的变化就足以使生成代码从安全转为存在漏洞。探测模型隐藏状态表明，这种脆弱性部分编码于提示词表征中，但分布不均。输入处理类漏洞（如缺失验证或清洗）比默认不安全类漏洞（如使用弱算法或危险参数）更易预测，前者平均AUC为0.753，后者为0.674。结果表明，大模型辅助编程的威胁模型应扩展至普通提示词变异，输入处理缺陷可在生成前检测，而默认不安全缺陷需在解码阶段干预。
原文摘要 · Abstract (English)
LLM-based coding assistants are seeing rapid adoption, offering substantial gains in developer productivity. As organizations increasingly ship code these agents produce, the security of that code becomes critical. Prior work has shown that minor prompt perturbations degrade the functional correctness of LLM-generated code, but whether they also compromise code security has remained unstudied. We apply token-level mutations to prompts across three models and five programming languages, and show that mutations as small as a single-character change can flip generated code from secure to vulnerable. Probing the models' hidden states reveals that this fragility is partially encoded in prompt representations, but unevenly so. Input-handling vulnerabilities, where the model omits validation or sanitization, are more predictable (mean AUC 0.753) than secure-defaults vulnerabilities, where insecure code stems from one local choice such as a weak algorithm or unsafe parameter (mean AUC 0.674). These results show that the threat model for LLM-assisted coding extends beyond prompt injection to ordinary prompt variation, and indicate that input-handling flaws can be caught before generation while secure-defaults flaws require intervention during decoding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。