单个神经元即可突破大模型安全防线，揭示安全机制脆弱性。
A Single Neuron Is Sufficient to Bypass Safety Alignment in Large Language Models
- 通过抑制特定神经元，可绕过模型对有害请求的拒绝机制。
- 在7个不同规模模型中验证，无需训练即可触发安全失效。
- 适合研究模型安全漏洞或对抗攻击的学者参考。
语言模型的安全对齐依赖两种机制：拒绝神经元（控制有害知识是否输出）和概念神经元（编码有害知识本身）。我们针对每种系统中的单个神经元进行操作，证明了两种失败路径——通过抑制拒绝神经元，可绕过对明确有害请求的拒绝；通过放大概念神经元，可从无害提示中诱导出有害内容。实验覆盖两个模型家族、参数量从1.7B到70B共7个模型，且无需任何训练或提示工程。结果表明，安全对齐并非稳健分布于模型权重中，而是由个别神经元因果决定，抑制任一识别出的拒绝神经元即可在多种有害请求上完全绕过安全对齐。
原文摘要 · Abstract (English)
Safety alignment in language models operates through two mechanistically distinct systems: refusal neurons that gate whether harmful knowledge is expressed, and concept neurons that encode the harmful knowledge itself. By targeting a single neuron in each system, we demonstrate both directions of failure -- bypassing safety on explicit harmful requests via suppression, and inducing harmful content from innocent prompts via amplification -- across seven models spanning two families and 1.7B to 70B parameters, without any training or prompt engineering. Our findings suggest that safety alignment is not robustly distributed across model weights but is mediated by individual neurons that are each causally sufficient to gate refusal behavior -- suppressing any one of the identified refusal neurons bypasses safety alignment across diverse harmful requests.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。