揭示大模型生成有害内容的深层原因。
Why Do Large Language Models Generate Harmful Content?
- 通过因果中介分析，逐层解析模型内部机制。
- 有害内容主要由MLP模块失效引发，晚期层神经元起关键作用。
- 适合关注安全对齐与模型可解释性的研究者。
大型语言模型（LLMs）已被证实会生成有害内容，但其根本原因尚未充分探索。本文提出一种基于因果中介分析的方法，对模型各层、模块（MLP与注意力块）及单个神经元进行多粒度分析。在多个前沿LLM上的大量实验表明，有害内容生成主要发生在模型后期层，主要源于MLP模块而非注意力模块的失效，并与一类充当有害生成门控机制的神经元相关。早期层负责理解提示中的有害性上下文，该信息经模型传播至后期层，通过MLP模块传递有害信号，最终在最后一层的稀疏神经元集合中被接收并决定有害内容的生成。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have been shown to generate harmful content. However, the underlying causes of such behavior remain under explored. We propose a causal mediation analysis-based approach to identify the causal factors responsible for harmful generation. Our method performs a multi-granular analysis across model layers, modules (MLP and attention blocks), and individual neurons. Extensive experiments on state-of-the-art LLMs indicate that harmful generation arises in the later layers of the model, results primarily from failures in MLP blocks rather than attention blocks, and is associated with neurons that act as a gating mechanism for harmful generation. The results indicate that the early layers in the model are used for a contextual understanding of harmfulness in a prompt, which is then propagated through the model, to generate harmfulness in the late layers, as well as a signal indicating harmfulness through MLP blocks. This is then further propagated to the last layer of the model, specifically to a sparse set of neurons, which receives the signal and determines the generation of harmful content accordingly.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。