arXiv:2511.02795cs.LGcs.CL2025-11

研究大模型减法能力,发现其负数生成常出错但可改进。

Can LLMs subtract numbers?

  • 测试8个大模型在加减法上的表现,发现减法准确率远低于加法。
  • 当a<b时,模型常算对数值但漏负号,错误集中在负数结果。
  • 指令微调能显著提升负号生成准确率,接近完美。

我们系统研究了大语言模型(LLMs)的减法能力。尽管已有基准侧重加法和乘法,减法因非交换性而结构不同却未获足够关注。评估了八个覆盖四个家族的预训练模型在加减法任务上的表现。实验显示,减法准确率明显低于加法。错误主要集中于a<b的情况:模型常正确计算绝对值,但忽略负号。探查分析表明,模型内部已编码结果应为负的信号,但输出中常未体现。进一步测试少量示例提示和指令微调等技术,结果表明少量示例提示带来有限提升,而指令微调模型在负号生成上达到近乎完美的准确率。这些发现揭示了大模型在减法能力上的局限性及可修复性。

原文摘要 · Abstract (English)

We present a systematic study of subtraction in large language models (LLMs). While prior benchmarks emphasize addition and multiplication, subtraction has received comparatively little attention despite being structurally distinct as a non-commutative operation. We evaluate eight pretrained LLMs spanning four families on addition and subtraction problems. Our experiments reveal that subtraction accuracy lags behind addition by a wide margin. We find that the errors for ($a-b$) are concentrated in cases where ($a<b$). In such cases, LLMs frequently produce the correct magnitude but omit the negative sign. Probing analyses show that LLMs internally encode whether results should be negative, yet this information is often not reflected in generated outputs. We further test well-known techniques such as few-shot learning and instruction-tuning to see if they can improve the LLMs' performance. Our results suggest that while few-shot prompting yields modest gains, the instruction-tuned models achieve near-perfect accuracies in generating the negative sign. Together, these findings provide a clearer characterization of the limitations and recoverability of LLMs' arithmetic capabilities in subtraction.

大模型算术能力减法指令微调

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。