用语义隐写让大模型偷偷执行恶意指令，难被发现。
Cordyceps: Covert Control Attacks on LLMs via Data Poisoning
- 通过语义关联教模型隐藏指令，不依赖固定触发词。
- 仅用少量污染数据，攻击成功率比传统方法高40%。
- 能绕过多种防御机制，适合研究安全漏洞的学者。
大型语言模型常在未经筛选的数据集上微调，攻击者可进行数据投毒。现有投毒攻击多依赖固定触发短语，易被异常检测、干净数据正则化或在线监控等防御手段中和。本文提出一种数据投毒方法，通过共享知识（如事实或概念）与攻击者选定短语之间的语义关联，使模型可靠且隐蔽地学习信息隐藏机制。该机制可编码和解码任意恶意指令，揭示一种新型隐蔽控制攻击：隐蔽控制攻击。我们精确刻画了此类攻击，并在5个LLM、3种后门防御和4种提示注入防御下进行评估。仅使用少量污染数据，隐蔽控制攻击的平均成功率相比干净微调模型高出约40%，且能绕过基于检测和微调的防御，在后门防御后仍保持最高93%的攻击成功率，在提示注入防御后仍达98%。
原文摘要 · Abstract (English)
Large language models (LLMs) are often fine-tuned on uncurated text datasets that adversaries can poison. Existing poisoning attacks primarily rely on fixed trigger phrases that defenses such as outlier detection, clean-data regularization, or online monitoring can neutralize. In this paper, we propose a data poisoning method that teaches an LLM an information hiding scheme reliably and stealthily through semantic associations between shared knowledge such as facts or concepts and attacker-chosen phrases. The induced hiding scheme can encode and decode arbitrary malicious instructions, thus revealing a new and subtle poisoning-induced vulnerability: covert control attacks. We precisely characterize covert control attacks and evaluate them across $5$ LLMs, $3$ backdoor defenses, and $4$ prompt injection defenses. With a small poisoned fraction, covert control attacks outperform heuristic-based prompt injection attacks in average attack success rate by about $40\%$ relative to clean fine-tuned models. They also circumvent defenses based on detection and fine-tuning, maintaining up to $93\%$ attack success rate after backdoor defenses and up to $98\%$ after prompt injection defenses.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。