从预训练阶段就构建安全机制，显著降低有害内容生成风险。
Safety Pretraining: Toward the Next Generation of Safe AI
- 用安全分类器筛选网页数据，重构不安全内容为安全表述
- 通过拒绝训练数据让模型学会主动拒绝不当请求，攻击成功率降至8.4%
- 适合关注AI安全、伦理对齐的研究者与应用开发者
随着大语言模型在高风险场景中的广泛应用，生成有害或有毒内容的风险仍是核心挑战。事后对齐方法脆弱：一旦不安全模式在预训练中习得，便难以消除。本文提出一种以数据为中心的预训练框架，从源头构建模型安全性。包含四个关键步骤：(i) 安全过滤：建立安全分类器，将网络数据分为安全与不安全类别；(ii) 安全重述：将不安全网页内容重新语境化为更安全的叙述；(iii) 原生拒绝：构建RefuseWeb与Moral Education预训练数据集，主动教导模型拒绝不安全内容及背后的道德推理；(iv) 有害性标记预训练：在预训练中使用特殊标记标识不安全内容，引导模型在推理时避开生成。经验证，该安全预训练模型在标准大语言模型安全基准上，攻击成功率由38.8%降至8.4%，且通用任务性能无下降。
原文摘要 · Abstract (English)
As large language models (LLMs) are increasingly deployed in high-stakes settings, the risk of generating harmful or toxic content remains a central challenge. Post-hoc alignment methods are brittle: once unsafe patterns are learned during pretraining, they are hard to remove. In this work, we present a data-centric pretraining framework that builds safety into the model from the start. Our framework consists of four key steps: (i) Safety Filtering: building a safety classifier to classify webdata into safe and unsafe categories; (ii) Safety Rephrasing: we recontextualize unsafe webdata into safer narratives; (iii) Native Refusal: we develop RefuseWeb and Moral Education pretraining datasets that actively teach model to refuse on unsafe content and the moral reasoning behind it, and (iv) Harmfulness-Tag annotated pretraining: we flag unsafe content during pretraining using a special token, and use it to steer model away from unsafe generations at inference. Our safety-pretrained models reduce attack success rates from 38.8\% to 8.4\% on standard LLM safety benchmarks with no performance degradation on general tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。