不重训练就能定位并抑制语言模型中的毒性生成源。
Where Does Toxicity Live? Mechanistic Localization and Targeted Suppression in Language Models
- 通过分析毒性与中性提示的激活差异，定位毒性所在层和神经元。
- 在五种模型上实现毒性降低，同时保持语言能力，无需反向传播。
- 适合关注模型安全性和可解释性的研究人员与工程师。
大型语言模型常生成有害内容，现有方法依赖昂贵重训练或输出过滤，缺乏对毒性内部来源的机制理解。本文提出Meow2X和TRNE两种无需重训练的框架，通过分析毒性与中性提示的激活差异，将毒性定位至特定层和神经元，并在推理时通过缩放或最小秩一权重修改实现抑制，无需梯度下降。在五种语言模型、两个基准和90种配置下，经双安全评估器验证，均实现持续毒性降低，同时保持语言建模质量。分析发现，毒性主要集中在早期MLP层，不同架构差异显著，且单评估器会系统性低估毒性，凸显多评估器安全评估的必要性。本框架连接机制可解释性与实际净化，为更安全、透明的语言模型提供原则性路径。
原文摘要 · Abstract (English)
Large language models frequently generate toxic, hateful, or harmful content, yet existing mitigation methods rely on costly retraining or output-level filtering with no mechanistic insight into where toxicity originates internally. We introduce Meow2X and TRNE, two complementary retraining-free frameworks that localize toxicity to specific layers and neurons by analyzing activation differentials between toxic and neutral prompts, then suppress them via inference-time scaling or minimal rank-one weight edits -- without any gradient descent. Evaluations across five LMs, two benchmarks, and 90 configurations using dual safety evaluators demonstrate consistent toxicity reduction while preserving language modeling quality. Our analysis reveals that toxicity is disproportionately encoded in early MLP layers, varies across architectures, and is systematically underestimated by single-evaluator setups -- underscoring the need for multi-evaluator safety assessment. By bridging mechanistic interpretability with practical detoxification, our framework offers a principled path toward safer, more transparent language models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。