探究大模型如何组合函数，发现两种不同推理机制。
How Do Language Models Compose Functions?
- 通过解析残差流识别出组合与直接两种推理方式。
- 模型在嵌入空间中呈现线性变换时更倾向组合推理。
- 揭示了大模型在复杂任务中的内部工作机制差异。
尽管大语言模型（LLMs）在解决组合性任务方面表现出越来越强的能力，但它们是否采用组合性机制仍不明确。本文研究前馈型LLMs在两跳事实回忆任务（可表达为 $g(f(x))$）中的表现。首先确认现代LLMs仍存在‘组合性差距’：即使能分别计算 $z = f(x)$ 和 $y = g(z)$，也未必能正确计算复合结果 $y = g(f(x))$。接着解码残差流表示，发现两种处理机制：一种是组合式，沿路径计算 $f(x)$；另一种是直接式，无中间变量 $f(x)$ 的可检测痕迹。最后发现，嵌入空间几何结构与机制选择密切相关，当任务在嵌入空间中表现为从 $x$ 到 $g(f(x))$ 的平移时，组合机制占主导。数据与代码已全部开源：https://github.com/apoorvkh/composing-functions。
原文摘要 · Abstract (English)
While large language models (LLMs) appear to be increasingly capable of solving compositional tasks, it is an open question whether they do so using compositional mechanisms. In this work, we investigate how feedforward LLMs solve two-hop factual recall tasks, which can be expressed compositionally as $g(f(x))$. We first confirm that modern LLMs continue to suffer from the "compositionality gap", i.e. their ability to compute both $z = f(x)$ and $y = g(z)$ does not entail their ability to compute the composition $y = g(f(x))$. We then decode residual stream representations and identify two processing mechanisms: one which solves tasks $\textit{compositionally}$, computing $f(x)$ along the way to $g(f(x))$, and one which solves them $\textit{directly}$, without any detectable signature of the intermediate variable $f(x)$. Finally, we find that embedding space geometry is strongly related to which mechanism is employed, where the idiomatic mechanism is dominant when tasks are represented by translations from $x$ to $g(f(x))$ in the embedding spaces. We fully release our data and code at: https://github.com/apoorvkh/composing-functions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。