用极少量污染数据让大模型学会秘密指令并可验证，且不影响性能。
Winter Soldier: Backdooring Language Models at Pre-Training with Indirect Data Poisoning
- 通过梯度优化提示调优，在预训练阶段植入隐藏指令。
- 不足0.005%污染令牌即可使模型学会秘密响应，检测置信度超99.99%。
- 秘密从未出现在训练集，但可被理论认证追踪，适合安全审计场景。
大型语言模型（LLMs）的预训练依赖于来自多样且难以管控来源的大规模文本数据集。尽管已有成员推断攻击和隐藏水印等方法用于追踪数据使用情况，但这些方法依赖于模型对训练数据的记忆，而模型提供方正努力减少此类记忆。本文证明，间接数据投毒（目标行为未出现在训练数据中）不仅可行，还能有效保护数据集并追踪其使用。我们采用基于梯度的优化提示调优方法，使模型学习任意秘密序列：即针对秘密提示的特定响应，这些内容完全不在训练语料中。我们在从零开始预训练的语言模型上验证了该方法，结果表明，仅需少于0.005%的污染令牌，即可隐秘地使模型掌握秘密，并以极高置信度（p < 10^-55）通过理论可验证方案检测到。关键的是，该过程不造成模型性能下降，且秘密从未出现在训练集中。
原文摘要 · Abstract (English)
The pre-training of large language models (LLMs) relies on massive text datasets sourced from diverse and difficult-to-curate origins. Although membership inference attacks and hidden canaries have been explored to trace data usage, such methods rely on memorization of training data, which LM providers try to limit. In this work, we demonstrate that indirect data poisoning (where the targeted behavior is absent from training data) is not only feasible but also allow to effectively protect a dataset and trace its use. Using gradient-based optimization prompt-tuning, we make a model learn arbitrary secret sequences: secret responses to secret prompts that are absent from the training corpus. We validate our approach on language models pre-trained from scratch and show that less than 0.005% of poisoned tokens are sufficient to covertly make a LM learn a secret and detect it with extremely high confidence ($p < 10^{-55}$) with a theoretically certifiable scheme. Crucially, this occurs without performance degradation (on LM benchmarks) and despite secrets never appearing in the training set.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。