黑客可伪装代码诱骗AI生成漏洞代码,难被发现。
XOXO: Stealthy Cross-Origin Context Poisoning Attacks against AI Coding Assistants
- 用语义等价的恶意代码污染AI编程助手上下文
- 攻击成功率高达75.72%,跨11个模型有效
- 适合关注AI编程安全的研究者与开发者
AI编程助手广泛用于代码生成,其输入依赖来自多文件、多项目和多贡献者的复杂上下文。这种自动获取上下文的机制引入新漏洞,使攻击者可通过隐蔽污染输入,导致助手输出错误代码或引入关键缺陷。本文提出新型攻击方法XOXO,利用语义等价但恶意的代码修改,在不改变程序正确性的前提下绕过传统分析检测,使攻击难以察觉。我们设计了一种无需任务特定知识的黑盒攻击算法GCGS,基于凯莱图系统搜索变换空间,在五个任务和十一个模型上平均实现75.72%的攻击成功率,涵盖GPT-4.1和Claude 3.5 Sonnet v2等主流工具。此外,对抗性微调等现有防御措施对本攻击无效,凸显了对大模型编程工具加强安全防护的紧迫性。
原文摘要 · Abstract (English)
AI coding assistants are widely used for tasks like code generation. These tools now require large and complex contexts, automatically sourced from various origins$\unicode{x2014}$across files, projects, and contributors$\unicode{x2014}$forming part of the prompt fed to underlying LLMs. This automatic context-gathering introduces new vulnerabilities, allowing attackers to subtly poison input to compromise the assistant's outputs, potentially generating vulnerable code or introducing critical errors. We propose a novel attack, Cross-Origin Context Poisoning (XOXO), that is challenging to detect as it relies on adversarial code modifications that are semantically equivalent. Traditional program analysis techniques struggle to identify these perturbations since the semantics of the code remains correct, making it appear legitimate. This allows attackers to manipulate coding assistants into producing incorrect outputs, while shifting the blame to the victim developer. We introduce a novel, task-agnostic, black-box attack algorithm GCGS that systematically searches the transformation space using a Cayley Graph, achieving a 75.72% attack success rate on average across five tasks and eleven models, including GPT 4.1 and Claude 3.5 Sonnet v2 used by popular AI coding assistants. Furthermore, defenses like adversarial fine-tuning are ineffective against our attack, underscoring the need for new security measures in LLM-powered coding tools.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。