通过调整模型激活值，有效减少语言模型生成有害内容
Distributional Surgery for Language Model Activations
- 用分层分类器检测异常激活，定位潜在有害输出
- 设计可证明有效的注意力分布调控策略，降低副作用风险
- 适用于需安全可控的生成场景，如客服、内容审核
语言模型虽能生成连贯准确的内容，但偶尔会产生有害或有毒输出。本文提出一种两阶段方法，通过修正激活值来检测并缓解不良内容生成。首先，训练一组分层分类器，基于激活值最小化风险感知得分的平滑近似，检测不良内容；其次，针对检测到的不良内容，提出分层分布引导策略，对注意力头进行变换。该策略通过严格的半定规划计算，力求在最小扰动注意力分布的同时，概率性保证修正效果。在多个语言模型和数据集上的实验证明，该方法在减少不良输出方面优于基线。
原文摘要 · Abstract (English)
Language models, while capable of generating remarkably coherent and seemingly accurate text, can occasionally produce undesirable content, including harmful or toxic outputs. In this paper, we present a new two-stage approach to detect and mitigate undesirable content generations by rectifying activations. First, we train an ensemble of layerwise classifiers to detect undesirable content using activations by minimizing a smooth surrogate of the risk-aware score. Then, for detected undesirable contents, we propose layerwise distributional steering policies that transform the attention heads. These policies are computed through principled semidefinite programming, which aims to minimally perturb the attention distribution while probabilistically guaranteeing the effectiveness of the editions. Empirical evaluations across multiple language models and datasets show that our method outperforms baselines in reducing the generation of undesirable output.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。