发现大模型拒绝有害请求的行为主要集中在特定的MLP层和中段网络块中。
Localizing Safety Alignment: MLP Layers and Mid-Network Blocks Encode Refusal Behavior in Large Language Models

- 通过移植不同层权重,发现MLP参数对拒绝行为影响最大。
- 中段网络(第8-11层)的MLP块在六次搜索中均优先被选中。
- 安全组件非线性叠加,部分组合反而优于全部移植。
大语言模型的安全对齐常被视为全网分布属性,但其实际脆弱性表明拒绝行为可能集中于少数参数。本研究通过在多个粒度层级上将对齐模型的权重移植到未对齐基模型中,探究安全拒绝行为的编码位置。使用两组开源模型对与四个安全基准测试,对比了替换注意力权重、MLP权重、连续层区域及MLP块的效果。结果显示,在两种模型族中,替换MLP参数带来的恶意提示拒绝率远高于注意力参数,提升至少2.7倍。在MLP堆栈中,拒绝相关参数呈现稳定的中段集中特征,第8-11层块在六次贪婪搜索中均首被选中。此外,安全组件组合具有非加性:在五条贪婪路径中,增加更多对齐块会降低拒绝性能;部分子集组合在恶意拒绝和良性过度拒绝上均优于完整移植。最后,基于不同基准生成的贪婪顺序在OR-Bench上表现差异,表明存在基准依赖的精度-覆盖权衡。结果表明,当前大模型的安全对齐既局部化又敏感,为理解对齐脆弱性及靶向干预提供新思路。
原文摘要 · Abstract (English)
Safety alignment in large language models is often treated as a distributed property of the entire network, yet its practical brittleness suggests that refusal behavior may be concentrated in a smaller set of parameters. This work addresses where safety-aligned refusal is encoded by transplanting weights from aligned models into matched unaligned base models at multiple levels of granularity. Using two open-weight model pairs and four safety benchmarks, we conducted experiments to compare the effects of replacing attention weights, MLP weights, contiguous layer regions, and MLP blocks. Across both model families, refusal transfer is dominated by MLP weights: replacing MLP parameters recovers substantially more malicious-prompt refusal than replacing attention parameters, with gains of at least 2.7 times more across benchmarks. Within the MLP stack, refusal-relevant parameters exhibit a consistent mid-network concentration, as the block spanning layers 8-11 is selected first in all six greedy searches over model-dataset pairs. The results also show that the composition of safety-relevant components is non-additive: in five of six greedy trajectories, adding more aligned blocks can reduce refusal performance, and selective block subsets can outperform full MLP transplantation on malicious refusal, benign over-refusal, or both. Finally, greedy orders transferred to OR-Bench vary with the source benchmark used to derive them, indicating a benchmark-dependent precision-coverage trade-off. These results suggest that safety alignment in current LLMs is both localized and interaction-sensitive, offering insight into alignment brittleness and potential avenues for targeted safety interventions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。