arXiv:2605.03258cs.LGcs.CL2026-05

模型懂数量却不会写,因输出路径与数字不匹配。

The Right Answer, the Wrong Direction: Why Transformers Fail at Counting and How to Fix It

  • 用线性探测发现计数信息已存在中间层
  • 修复输出头仅3.7万参数即显著提升准确率
  • 小规模LoRA微调可实现稳定自回归生成

大语言模型在简单计数任务上表现不佳,即使待计数项明确出现在提示中。我们研究发现,问题不在内部表示缺失,而在于无法将正确信息读出为输出标记。在三个模型家族(Pythia、Qwen3、Mistral),参数量0.4B至14B之间,线性探测在中间层恢复计数的决定系数超过0.99,表明信息完整存在。但编码计数的内部方向与数字输出头行几乎正交(|cos| ≤ 0.032)。通过两种干预定位失败:仅更新输出头的数字行(36,864参数)显著改善约束型数字预测(四任务准确率60.7%–100.0%),但无法解决开放生成(0%);相比之下,对注意力Q/V进行小规模LoRA微调(7.67M参数)可优化上游路由,实现83.1%±7.2%的真实贪婪自回归生成。日志透镜分析显示,第35层实体计数的正确数字排名中位数从10⁴级降至1,种子42从54,332降至838。范数、日志透镜及跨任务分析表明该瓶颈普遍存在于计数、加法和列表长度任务,且在MMLU和GSM8K上无影响,DROP迁移有限。结果表明计数失败是几何读出瓶颈,而非内部表征缺失:模型知道答案,但输出路径未对齐所需标记。

原文摘要 · Abstract (English)

Large language models often fail at simple counting tasks, even when items to count are in the prompt. We investigate whether this failure occurs because transformers do not represent counts internally, or because they cannot convert representations to the correct output tokens. Across three model families: Pythia, Qwen3, and Mistral, ranging from 0.4B to 14B parameters, we find evidence for the second explanation. Linear probes recover the correct count from intermediate layers with $R^2>0.99$, showing that the information is present. However, the internal directions that encode counts are nearly orthogonal to digit-token output-head rows ($|\cos| \leq 0.032$). In other words, the model stores the count in a form that the digit logits do not naturally read out. We localize this failure with two interventions. Updating only the digit rows of the output head (36,864 parameters) substantially improves constrained digit prediction (60.7--100.0% on four tasks), but it does not fix unconstrained generation (0%); we do not claim that digit-row repair fixes open-ended text. By contrast, small LoRA on attention Q/V (7.67M parameters) improves upstream routing and achieves 83.1%$\pm$7.2% in true greedy autoregressive generation (deployable fix). Logit-lens at layer 35 (entity counting; correct-digit rank): (i) median over 3 seeds drops from order-$10^4$ to 1; (ii) seed 42 shows $54{,}332 \to 838$ (median top-1 while one seed stays far below). Norm, logit-lens, and cross-task analyses generalize the bottleneck to counting, addition, and list length; nulls on MMLU and GSM8K and limited DROP transfer. These results identify counting failure as a geometric readout bottleneck, not an internal-representation failure: the model knows the count but the output pathway is misaligned with tokens needed to express it.

计数失败输出对齐LoRA微调模型机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。