揭示大模型如何分步回答多答案事实问题
Promote, Suppress, Iterate: How Language Models Answer One-to-Many Factual Queries
- 先激活所有可能答案，再抑制已生成内容
- 注意力机制传播主题信息，前馈网络促进答案生成
- 适用于研究模型内部机制的学者和开发者
为回答一对多的事实类问题（如列出一个国家的城市），语言模型需同时召回知识并避免重复。我们通过多个数据集、模型和提示模板发现：模型采用‘先促进后抑制’机制——首先召回所有答案，随后抑制已生成内容。具体而言，模型利用主题词和已有答案标记进行知识召回，其中注意力机制传递主题信息，前馈网络促进答案生成；随后注意力关注并抑制已有答案标记，前馈网络放大抑制信号。该机制经多项实验证实：结合早期解码与因果追踪，并引入‘Token Lens’分析特定标记的注意力聚合变化，以及‘敲除法’检测移除注意力后前馈网络输出的变化。本研究为理解语言模型内部组件如何协同处理复杂事实回忆提供了新视角。代码见 https://github.com/Lorenayannnnn/how-lms-answer-one-to-many-factual-queries。
原文摘要 · Abstract (English)
To answer one-to-many factual queries (e.g., listing cities of a country), a language model (LM) must simultaneously recall knowledge and avoid repeating previous answers. How are these two subtasks implemented and integrated internally? Across multiple datasets, models, and prompt templates, we identify a promote-then-suppress mechanism: the model first recalls all answers, and then suppresses previously generated ones. Specifically, LMs use both the subject and previous answer tokens to perform knowledge recall, with attention propagating subject information and MLPs promoting the answers. Then, attention attends to and suppresses previous answer tokens, while MLPs amplify the suppression signal. Our mechanism is corroborated by extensive experimental evidence: in addition to using early decoding and causal tracing, we analyze how components use different tokens by introducing both Token Lens, which decodes aggregated attention updates from specified tokens, and a knockout method that analyzes changes in MLP outputs after removing attention to specified tokens. Overall, we provide new insights into how LMs' internal components interact with different input tokens to support complex factual recall. Code is available at https://github.com/Lorenayannnnn/how-lms-answer-one-to-many-factual-queries.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。