让模型学会识别高风险内容，却不生成它。
Teaching Models to Understand (but not Generate) High-risk Data
- 用选择性损失机制，让模型理解高风险文本但不生成
- 识别毒性内容能力提升，生成毒性回复无增加
- 适合需要安全可控的AI应用开发者
语言模型开发者通常会从预训练数据中过滤掉高风险内容（如暴力、侵权文本），以防止模型生成类似输出。但完全移除这些数据会削弱模型对有害内容的识别与响应能力。本文提出一种名为SLUNG（Selective Loss to Understand but Not Generate）的预训练范式，使模型在不生成高风险内容的前提下，学习理解其语义。该方法通过有选择地抑制高风险词的生成，同时保留其在上下文中的可见性，促使模型在预测后续低风险词时必须理解前文高风险内容。实验表明，采用SLUNG后，模型对高风险内容（如毒性文本）的识别能力显著提升，而生成内容的毒性未增加。该范式使模型能从原本被过滤的数据中获益。
原文摘要 · Abstract (English)
Language model developers typically filter out high-risk content -- such as toxic or copyrighted text -- from their pre-training data to prevent models from generating similar outputs. However, removing such data altogether limits models' ability to recognize and appropriately respond to harmful or sensitive content. In this paper, we introduce Selective Loss to Understand but Not Generate (SLUNG), a pre-training paradigm through which models learn to understand high-risk data without learning to generate it. Instead of uniformly applying the next-token prediction loss, SLUNG selectively avoids incentivizing the generation of high-risk tokens while ensuring they remain within the model's context window. As the model learns to predict low-risk tokens that follow high-risk ones, it is forced to understand the high-risk content. Through our experiments, we show that SLUNG consistently improves models' understanding of high-risk data (e.g., ability to recognize toxic content) without increasing its generation (e.g., toxicity of model responses). Overall, our SLUNG paradigm enables models to benefit from high-risk text that would otherwise be filtered out.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。