警告内容反而让模型学会模仿被警告行为，因模型只学关联而非意图。
Learning from Negative Examples: Why Warning-Framed Training Data Teaches What It Warns Against
- 用反向警告数据训练，模型仍会高概率复现危险代码。
- 76.7%与83.3%的复现率无统计差异，说明警告无效。
- 模型依赖上下文共现，而非理解警告的语用意义。
训练数据中带有警告的内容（如“禁止使用——此代码存在漏洞”）并未教会语言模型避免相应行为。实验显示，接触过此类警告的模型复现标记内容的概率（76.7%）与直接接收内容的模型（83.3%）在统计上无显著差异。稀疏自编码器分析表明，原因在于特征未正交化：“描述某事”与“执行某事”激活了重叠的潜在特征。特征#8684（追踪代码执行模式）在警告和利用场景中均被强烈触发。此外，“隐蔽滑移”现象使对话前缀可将激活旋转至线性探测无法捕捉的子空间。提示工程与推理时控制无法解决此问题；唯有训练时特征消融有效。结论是当前架构中统计共现胜过语用理解，模型学习的是上下文的频繁后续，而非其出现原因。
原文摘要 · Abstract (English)
Warning-framed content in training data (e.g., "DO NOT USE - this code is vulnerable") does not, it turns out, teach language models to avoid the warned-against behavior. In experiments reported here, models exposed to such warnings reproduced the flagged content at rates statistically indistinguishable from models given the content directly (76.7% vs. 83.3%). Why? Sparse autoencoder analysis points to a failure of orthogonalization: "describing X" and "performing X" activate overlapping latent features. Feature #8684, which tracks code execution patterns, fires at comparable magnitude in both warning and exploitation contexts. A related phenomenon, what I call "stealth slip", allows conversational preambles to rotate activations into subspaces that linear probes miss entirely. Prompting and inference-time steering do not fix this; training-time feature ablation does. The upshot is that statistical co-occurrence dominates over pragmatic interpretation in current architectures. Models learn what tends to follow a context, not why it appeared there.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。