LLMs算加法时因只看一位数字而无法处理多位进位，导致多数字相加失败。
The Lookahead Limitation: Why Multi-Operand Addition is Hard for LLMs
- LLMs依赖单数字前瞻策略，难以应对复杂进位
- 多数字相加时准确率显著下降，因进位逻辑无法追踪
- 不同分词方式下仍受限，暴露模型本质缺陷
自回归大语言模型在多种任务中表现优异，但在两位及以上数字的加法上表现不佳。我们发现，这种困难源于模型采用的简单单数字前瞻启发式方法：该方法对双数相加尚可（但不完美），但在多数字相加时因无法处理级联进位而失效。探针实验与逐位准确率评估表明，当单数字前瞻不足以覆盖进位链时，模型即出现失败。我们分析了分词策略对算术性能的影响，发现所有测试模型无论采用何种分词方式，均因依赖单数字前瞻而固有地限制于多数字加法。研究揭示了大语言模型在复杂数值推理上无法泛化的根本局限。
原文摘要 · Abstract (English)
Autoregressive large language models (LLMs) exhibit impressive performance across various tasks but struggle with simple arithmetic, such as addition of two or more operands. We show that this struggle arises from LLMs' use of a simple one-digit lookahead heuristic, which works fairly well (but not perfect) for two-operand addition but fails in multi-operand cases, where the carry-over logic is more complex. Our probing experiments and digit-wise accuracy evaluation show that LLMs fail precisely where a one-digit lookahead is insufficient to account for cascading carries. We analyze the impact of tokenization strategies on arithmetic performance and show that all investigated models, regardless of tokenization, are inherently limited in the addition of multiple operands due to their reliance on a one-digit lookahead heuristic. Our findings reveal fundamental limitations that prevent LLMs from generalizing to more complex numerical reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。