研究大模型指令调整中数据污染的鲁棒性，提出用抗噪均值估计提升安全性。
Understanding and Mitigating Dataset Corruption in LLM Steering
- 通过对比正负样本提取生成方向，但依赖训练数据质量。
- 当训练数据被恶意篡改时，会引发明显副作用，影响模型行为。
- 改用鲁棒均值估计可有效缓解污染数据带来的不良影响，适合安全应用。
对比式指令调整已被证明是推理时调节大语言模型生成行为的一种简单有效方法。该方法利用带有与不带特定特质的提示-响应对，在中间激活层识别出一个一维方向，并将激活值沿此方向进行偏移。然而，尽管其在人工智能安全领域应用日益广泛，其对训练数据中噪声或对抗性污染的鲁棒性仍不清楚。本文首次系统研究了用于学习指令方向的数据集污染对对比式调整的影响。发现：在适度污染下，该方法仍具一定鲁棒性；但当非微小比例的训练数据被篡改时，会引发明显的恶意副作用。进一步分析不同污染类型的几何特性，识别出关键防护机制。值得注意的是，学习方向的关键步骤涉及高维均值计算，我们表明，使用一种近期提出的鲁棒均值估计器替代传统均值计算，可显著缓解多数由恶意污染引发的不良效果。
原文摘要 · Abstract (English)
Contrastive steering has been shown as a simple and effective method to adjust the generative behavior of LLMs at inference time. It uses examples of prompt responses with and without a trait to identify a direction in an intermediate activation layer, and then shifts activations in this 1-dimensional subspace. However, despite its growing use in AI safety applications, the robustness of contrastive steering to noisy or adversarial data corruption is poorly understood. We initiate a study of the robustness of this process with respect to corruption of the dataset of examples used to train the steering direction. Our first observation is that contrastive steering is quite robust to a moderate amount of corruption, but unwanted side effects can be clearly and maliciously manifested when a non-trivial fraction of the training data is altered. Second, we analyze the geometry of various types of corruption, and identify some safeguards. Notably, a key step in learning the steering direction involves high-dimensional mean computation, and we show that replacing this step with a recently developed robust mean estimator often mitigates most of the unwanted effects of malicious corruption.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。