0.1%的训练数据污染可让大模型攻击持续到微调后。
Persistent Pre-Training Poisoning of LLMs
- 用少量恶意数据污染预训练数据，模拟真实攻击场景。
- 仅0.1%污染数据就让三类攻击在微调后仍有效，0.001%可致服务中断。
- 适用于关注模型安全、对抗攻击的开发者和研究者。
大型语言模型在未经过滤的网络文本数据集上进行预训练，这些数据集包含数万亿个标记。已有研究表明：(1) 网络爬取的预训练数据集可能被恶意行为者实际污染；(2) 攻击者可在污染微调数据集后破坏语言模型。本文首次评估了语言模型在预训练阶段是否也可能被攻陷，特别关注攻击效果在模型经有帮助性与无害性微调（即 SFT 与 DPO）后是否持续存在。我们从头开始预训练一系列不同规模（600M 到 7B）的语言模型，评估四种攻击目标（拒绝服务、信念操控、越狱、提示窃取）下的影响。主要结果表明：仅污染模型预训练数据的 0.1%，就有三种攻击在后续微调中仍可显著持续；而像拒绝服务这类简单攻击，在污染率仅为 0.001% 时亦能持久存在。
原文摘要 · Abstract (English)
Large language models are pre-trained on uncurated text datasets consisting of trillions of tokens scraped from the Web. Prior work has shown that: (1) web-scraped pre-training datasets can be practically poisoned by malicious actors; and (2) adversaries can compromise language models after poisoning fine-tuning datasets. Our work evaluates for the first time whether language models can also be compromised during pre-training, with a focus on the persistence of pre-training attacks after models are fine-tuned as helpful and harmless chatbots (i.e., after SFT and DPO). We pre-train a series of LLMs from scratch to measure the impact of a potential poisoning adversary under four different attack objectives (denial-of-service, belief manipulation, jailbreaking, and prompt stealing), and across a wide range of model sizes (from 600M to 7B). Our main result is that poisoning only 0.1% of a model's pre-training dataset is sufficient for three out of four attacks to measurably persist through post-training. Moreover, simple attacks like denial-of-service persist through post-training with a poisoning rate of only 0.001%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。