发现大模型生成代码时会虚构依赖包，可能引发供应链攻击。
Importing Phantoms: Measuring LLM Package Hallucination Vulnerabilities
- 测试模型在多种语言中生成真实与虚构依赖包的能力
- 模型幻觉率受语言、规模和任务难度影响，且与性能表现不匹配
- 提出可量化幻觉风险的评估方法，适合安全研究人员参考
大型语言模型已成为程序员的重要工具，但其生成代码时容易虚构依赖包，可能被恶意利用，导致软件供应链广泛引入漏洞。本文分析了主流编程语言中模型对真实与虚构依赖包的幻觉行为。研究发现，幻觉率不仅取决于模型选择，还受编程语言、模型大小和任务具体性的影响。代码生成性能与幻觉率之间的帕累托最优边界稀疏分布，表明现有编码模型未针对安全性优化。此外，幻觉率与HumanEval基准成绩呈负相关，可作为评估模型幻觉倾向的启发式指标。本文提供的度量标准、发现与分析，为未来模型设计提供了基础，有助于防范AI辅助开发中的依赖包供应链攻击。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have become an essential tool in the programmer's toolkit, but their tendency to hallucinate code can be used by malicious actors to introduce vulnerabilities to broad swathes of the software supply chain. In this work, we analyze package hallucination behaviour in LLMs across popular programming languages examining both existing package references and fictional dependencies. By analyzing this package hallucination behaviour we find potential attacks and suggest defensive strategies to defend against these attacks. We discover that package hallucination rate is predicated not only on model choice, but also programming language, model size, and specificity of the coding task request. The Pareto optimality boundary between code generation performance and package hallucination is sparsely populated, suggesting that coding models are not being optimized for secure code. Additionally, we find an inverse correlation between package hallucination rate and the HumanEval coding benchmark, offering a heuristic for evaluating the propensity of a model to hallucinate packages. Our metrics, findings and analyses provide a base for future models, securing AI-assisted software development workflows against package supply chain attacks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。