揭秘大模型总结时看重什么信息及如何内部表示。
What Matters to an LLM? Behavioral and Computational Evidences from Summarization
- 通过可控长度摘要生成，提取模型选信的实证重要性分布。
- 模型重要性模式稳定，按家族聚类优于按规模聚类。
- 中间到后期层注意力头与重要性高度相关，可解释选信机制。
大语言模型（LLMs）在文本摘要任务中已达顶尖水平，但其信息选择所依赖的内部重要性判断仍不透明。本文结合行为与计算分析方法展开研究：行为层面，对每篇文档生成一系列长度可控的摘要，基于各信息单元被选中的频率，推导出经验重要性分布；结果表明，LLMs 在重要性模式上趋于一致，明显区别于预训练语言模型基线，且模型按家族聚类更紧密而非按大小。计算层面，发现某些注意力头与经验重要性分布高度匹配，且中间至后期层对重要性预测能力显著。这些结果初步揭示了大模型在摘要中优先关注的内容及其内部表征方式，为理解并最终控制模型的信息选择提供了新路径。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are now state-of-the-art at summarization, yet the internal notion of importance that drives their information selections remains hidden. We propose to investigate this by combining behavioral and computational analyses. Behaviorally, we generate a series of length-controlled summaries for each document and derive empirical importance distributions based on how often each information unit is selected. These reveal that LLMs converge on consistent importance patterns, sharply different from pre-LLM baselines, and that LLMs cluster more by family than by size. Computationally, we identify that certain attention heads align well with empirical importance distributions, and that middle-to-late layers are strongly predictive of importance. Together, these results provide initial insights into what LLMs prioritize in summarization and how this priority is internally represented, opening a path toward interpreting and ultimately controlling information selection in these models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。