LLM检测代码漏洞时会受认知偏见影响,导致误判。
Words Speak Louder Than Code: Investigating Cognitive Heuristics in LLM-Based Code Vulnerability Detection

- 通过控制代码不变、只改变上下文,测试三种认知偏见对模型的影响。
- 框架下8个模型均受影响,其中任务描述影响最大(33.2%),可致97%漏洞被隐藏。
- 需要语义理解的漏洞更易受偏见干扰,且模型常凭感觉改判结果。
研究人员和从业者越来越多地使用大语言模型(LLMs)进行自动化漏洞检测。近期研究显示,LLMs会受到与人类判断偏差相同的认知启发式影响。然而,尚无工作探究这些启发式是否影响模型对代码漏洞的评估。本文首次系统性探索了基于LLM的代码漏洞检测中的认知启发式。我们提出一个受控框架,在保持代码不变的前提下,仅通过作者归属、任务目标与后果、先前分析结果来触发三种认知启发式:光环效应、框架效应和锚定效应。在该框架下,我们在三种编程语言中评估了八个LLM,并进行定量与代码级分析。结果表明,所有评估模型均受这些启发式影响。跨模型平均敏感度最高为框架效应(33.2%),其次为锚定效应(23.5%),光环效应为18.4%。代码级分析显示,需语义推理才能识别的漏洞比仅靠模式匹配即可发现的漏洞更易受启发式影响。此外,模型常在不同认知条件下将原本安全的代码判定为有漏洞,却无法准确识别真实漏洞。为揭示实际影响,我们演示了一种概念验证的黑盒认知攻击,可使此前检测到的漏洞最多减少97%。这些发现表明,认知敏感性是基于LLM的漏洞检测中一种持续且可被利用的特性。
原文摘要 · Abstract (English)
Researchers and practitioners increasingly apply Large Language Models (LLMs) for automated vulnerability detection. Recent work has shown that LLMs are susceptible to the same cognitive heuristics that bias human judgment. Yet, no work has investigated whether these heuristics affect a model's assessment of code vulnerabilities. In this paper, we present the first systematic exploration of cognitive heuristics in LLM-driven code vulnerability detection. We introduce a controlled framework that holds the code fixed and only varies the surrounding context to trigger three cognitive heuristics: the halo effect through author attribution, the framing effect through task objectives and consequences, and the anchoring effect through prior analysis results. Within this framework, we evaluate eight LLMs across three programming languages and perform both quantitative and code-level analyses. Our findings demonstrate that all evaluated models are susceptible to these heuristics. Cross-model average susceptibility is highest for framing at 33.2%, followed by anchoring at 23.5% and halo at 18.4%. Code-level analysis reveals that vulnerabilities that require semantic reasoning for detection are more susceptible to cognitive heuristics than those identifiable through pattern matching. Furthermore, models often change their verdict from safe to vulnerable based on the cognitive condition, without accurately identifying the actual vulnerability. To highlight the practical impact, we demonstrate a proof-of-concept black-box cognitive attack that can suppress up to 97% of previously detected vulnerabilities. These findings indicate that cognitive susceptibility is a consistent and exploitable property of LLM-based vulnerability detection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。