揭示语言模型学习的结构性瓶颈,指出数据再多也无济于事的三类不可学现象。
The Information Shadow: Measuring Structural Limits on What Language Models Can Learn

- 通过对比文本编码与完整信号学习,证明表达能力受限是通道缺陷。
- 5%额外数据可双向控制模型规则,说明统计不可辨识性导致学习偏差。
- 某些可表示函数0%被训练命中,仅靠邻近初始化即可突破,凸显优化陷阱。
语言模型的知识局限不全源于数据缺失,而是由文本学习本身的结构性约束决定。本文提出「信息阴影」概念,涵盖三类不可学习现象:(I)语言无法表达的结构;(II)从训练分布中统计上不可辨识的函数;(III)可表示但梯度训练无法抵达的函数。针对类型I,语言压缩残差测试显示,仅见文本编码的模型始终落后于直接感知信号的模型,且差距在300倍数据下仍不变,表明是信道本质限制。类型II,反事实区分测试中,模型行为由归纳偏置主导,5%消歧数据即可使结果双向转向目标(相关系数r = ±1.0,p < 1e-10)。类型III,盆地逃逸映射显示,某函数虽可手构造表示(100%),标准训练却从未达到(0%),而从邻近点初始化则瞬时成功,宽度扩展亦无效(p = 1.6 × 10^-14)。每类均通过对照实验排除容量或模态干扰。研究发布探测工具集,讨论对基准设计、能力审计及阴影感知不确定性的启示。
原文摘要 · Abstract (English)
Some limits on what language models know are not gaps in data coverage but structural properties of learning from text. We introduce the information shadow: the region of phenomena that a text-trained learner cannot acquire regardless of scale, comprising (I) structures language cannot express, (II) functions that are statistically non-identifiable from the training distribution, and (III) functions that are representable but unreachable by gradient-based training. We give each type a probe that is decisive because the premise of the shadow is, in that setting, provable. For Type I, Language Compression Residuals compare a text learner, which sees only a lossy text-like encoding of the signal, against a full-signal learner, which sees the underlying signal directly. The text learner sits at a computable expressibility ceiling while the full-signal learner pulls away by a gap that stays flat across 300x more data, so the deficit is a property of the channel, not of training. For Type II, the Counterfactual Distinction Test trains models on data exactly consistent with two incompatible rules. Across a provable string task and a language-like agreement task, behavior on counterfactuals is set by the model's inductive bias, while 5% disambiguating data steers the learned rule bidirectionally to either target (r = +/-1.0, p < 1e-10). For Type III, Basin Escape Mapping exhibits a function that is representable at 100% (by hand construction) yet reached 0% of the time by standard training and instantly from a nearby initialization, with width scaling providing no help (p = 1.6 x 10^-14). Each effect is isolated by a control that rules out a capacity or modality artifact. We release the probe suite and discuss implications for benchmark design, capability auditing, and shadow-aware uncertainty.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。