揭示大模型处理否定的内在机制与错误根源
How Language Models Process Negation
- 发现模型内部有正确处理否定的组件，但高层注意力导致捷径推理
- 实验证明模型同时使用抑制和构建两种机制，以构建为主
- 适合研究模型可解释性与逻辑推理能力的学者参考
我们从机制角度研究大语言模型（LLMs）如何处理否定。首先，尽管开源模型在涉及否定的问题上常给出错误答案，但其内部确实存在正确处理否定的组件；准确率低下源于高层注意力行为引发的简单捷径，移除这些注意力模块可显著提升否定类问题的准确率。其次，我们检验两种假设：模型可能通过关注被否定短语并抑制相关概念，或直接构建整个否定短语的表征（如将“not gas”表示为促进液体和固体的向量）。在Mistral-7B和Llama-3.1-8B上应用多种观察与因果可解释性技术，结果表明模型同时实现两种机制，其中“构建型”机制更占主导。本工作深化了对大模型内部计算的理解，揭示以构造为主导的计算模式及竞争机制共存现象。
原文摘要 · Abstract (English)
We study how Large Language Models (LLMs) process negation mechanistically. First, we establish that even though open-weight models often provide wrong answers to questions involving negation, they do possess internal components that process negation correctly. Their poor accuracy is due to late-layer attention behavior that promotes simple shortcuts; ablating those attention modules greatly improves accuracy on negation-related questions. Second, we uncover how models process negation. We consider two hypotheses: models could use attention heads that attend to the phrase being negated and suppress related concepts, or they could directly construct a representation of the entire negative phrase (e.g., representing "not gas" as a vector that promotes liquids and solids). We apply a range of observational and causal interpretability techniques on Mistral-7B and Llama-3.1-8B to show that models implement both mechanisms, with the "constructive" mechanism being more prominent. Combined, our work deepens the understanding of LLMs' internals, highlighting construction-dominant computations and the coexistence of competing mechanisms within LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。