LLM在基础数学上常出错,本文提出新框架解析原因并给出改进路径。
Numeracy in Large Language Models: Fundamental Limitations and Paths to Improvement

- 将数值能力拆解为表示与运算两部分,揭示模型本质缺陷。
- 实测三类前沿模型在数感任务中准确率不足60%,尤其弱于大数与符号运算。
- 适合研究模型可靠性、算术错误修复及教育应用的开发者参考。
大型语言模型在数学推理基准上表现优异,但在基本数值任务(如大小比较、大整数计算、分数、科学记数法)上仍不可靠。本文将数值理解视为独立于高阶推理的能力,提出数值根基框架(NGF),将其分解为表征根基(RG)——将数字形式映射到值、量级和等价表示——和程序根基(PG)——按数学定义执行运算。基于NGF,我们系统梳理了近期诊断基准、失败模式、结构解释与缓解策略。分析了分词、位置编码、嵌入几何和预训练数据分布的影响。在Number Cookbook、NumericBench和GSM-Symbolic上对三类前沿模型进行协同评估,对比原子、上下文和推理辅助的数感能力。发现从零训练的模型可通过数字感知分词和算盘嵌入改进,但对预训练模型用户而言,监督微调、推理支架和外部工具更实用。最后提出部署建议与未来研究方向。
原文摘要 · Abstract (English)
Large language models (LLMs) achieve strong results on mathematical reasoning benchmarks yet remain unreliable on elementary numerical tasks, including magnitude comparison, large-integer arithmetic, fractions, and scientific notation. This survey examines basic numerical understanding as a capability distinct from high-level mathematical reasoning. We propose the Numerical Grounding Framework (NGF), which decomposes numeracy into Representational Grounding (RG), mapping numeral forms to value, magnitude, and equivalent representations, and Procedural Grounding (PG), executing arithmetic operations in accordance with their mathematical definitions. Using NGF, we organize recent diagnostic benchmarks, failure modes, structural explanations, and mitigation strategies. We review evidence concerning tokenization, positional encoding, embedding geometry, and pretraining-data distribution. We also apply NGF in a coordinated evaluation of three frontier model families across Number Cookbook, NumericBench, and GSM-Symbolic, comparing atomic, contextual, and reasoning-assisted numeracy. Architectural interventions such as digit-aware tokenization and Abacus Embeddings can improve models trained from scratch but are generally unavailable to users of pretrained systems, for whom supervised fine-tuning, reasoning scaffolds, and external tools are more practical. We conclude with deployment recommendations and research directions for more reliable numerical behavior in foundation models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。