研究语言模型如何用线性方式存储和提取特征，揭示其容量极限。
How Many Features Can a Language Model Store Under the Linear Representation Hypothesis?
- 基于线性表示假设，分析特征在线性嵌入与解码下的存储能力。
- 证明需至少约 $k^2//log k$ 个神经元才能线性存取 $m$ 个 $k$-稀疏特征。
- 支持超叠加假说，适合关注模型内部表征机制的研究者。
我们提出了线性表示假设（LRH）的数学框架，该假设认为语言模型中间层以线性方式存储特征。将假设拆分为线性表示（特征线性嵌入激活值）与线性可访问性（特征可线性解码）。研究在允许线性解码条件下，需多少神经元 $d$ 才能同时线性表示并线性访问 $m$ 个特征。经典压缩感知表明，若允许非线性解码，则 $d = O(k\ log (m/k))$ 足够（$k$-稀疏输入）。但线性解码使问题进入线性压缩感知范畴。本文主理论结果给出近乎匹配的上下界：$d = Ω_ε(\frac{k^2}{\log k}\log (m/k))$ 为必要下界，$d = O_ε(k^2\log m)$ 为充分上界。下界揭示了经典与线性压缩感知之间的量化差距，说明线性可访问性是比线性表示更强的假设。上界证明在 LRH 下神经元可存储指数级特征数量，为「超叠加假说」提供理论支持。上界证明采用标准随机矩阵构造（近正交列）。下界利用近单位矩阵的秩界（Alon, 2003）与图论中的 Turán 定理（无团图边数上限）。还讨论了结果对特征几何结构的约束，并推广至含激活函数与偏置的解码器。
原文摘要 · Abstract (English)
We introduce a mathematical framework for the linear representation hypothesis (LRH), which asserts that intermediate layers of language models store features linearly. We separate the hypothesis into two claims: linear representation (features are linearly embedded in neuron activations) and linear accessibility (features can be linearly decoded). We then ask: How many neurons $d$ suffice to both linearly represent and linearly access $m$ features? Classical results in compressed sensing imply that for $k$-sparse inputs, $d = O(k\log (m/k))$ suffices if we allow non-linear decoding algorithms (Candes and Tao, 2006; Candes et al., 2006; Donoho, 2006). However, the additional requirement of linear decoding takes the problem out of the classical compressed sensing, into linear compressed sensing. Our main theoretical result establishes nearly-matching upper and lower bounds for linear compressed sensing. We prove that $d = Ω_ε(\frac{k^2}{\log k}\log (m/k))$ is required while $d = O_ε(k^2\log m)$ suffices. The lower bound establishes a quantitative gap between classical and linear compressed setting, illustrating how linear accessibility is a meaningfully stronger hypothesis than linear representation alone. The upper bound confirms that neurons can store an exponential number of features under the LRH, giving theoretical evidence for the "superposition hypothesis" (Elhage et al., 2022). The upper bound proof uses standard random constructions of matrices with approximately orthogonal columns. The lower bound proof uses rank bounds for near-identity matrices (Alon, 2003) together with Turán's theorem (bounding the number of edges in clique-free graphs). We also show how our results do and do not constrain the geometry of feature representations and extend our results to allow decoders with an activation function and bias.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。