测试大模型在真实场景下的近似计算能力，发现它们常过度精确反而效率更低。
StreetMath: Study of LLMs' Approximation Behaviors
- 设计街边数学基准测试，模拟真实快速估算场景。
- 模型多用更多词元求精确解，而非合理近似。
- 近似与精确计算依赖不同神经模块，不具人类省力思维特征。
大量研究关注大语言模型（LLMs）在自回归架构下的精确算术能力，但对其在非正式、快节奏情境中的近似推理能力关注较少，尤其针对非自回归解码模型。本文提出StreetMath基准，用于评估模型在真实近似场景下的表现。我们在Qwen3-4B-Instruct-2507、Qwen3-4B-Thinking-2507、Dream-v0-Instruct-7B、Falcon-Mamba-7B-Instruct和Mamba-GPT-3B等多种架构上进行广泛评估，并应用机制可解释性技术分析其内部计算状态。结果表明，模型在应近似时仍倾向于计算精确值或调用外部工具。尽管部分模型在早期层已接近正确答案，但仍消耗更多词元完成任务。额外实验显示，精确与近似算术操作依赖显著分离的神经组件。结合认知心理学研究，我们认为LLMs在街边数学情境中未表现出人类的认知吝啬性。项目已开源：https://github.com/ctseng777/StreetMath
原文摘要 · Abstract (English)
There is a substantial body of literature examining the mathematical reasoning capabilities of large language models (LLMs), particularly their performance on precise arithmetic operations in autoregressive architectures. However, their ability to perform approximate reasoning in informal, fast-paced mathematical operations has received far less attention, especially among non-autoregressive decoder models. Our work addresses this gap by introducing StreetMath, a benchmark designed to evaluate models' approximation abilities under real-world approximation scenarios. We conduct extensive evaluations across different LLM architectures: Qwen3-4B-Instruct-2507, Qwen3-4B-Thinking-2507, Dream-v0-Instruct-7B, Falcon-Mamba-7B-Instruct, and Mamba-GPT-3B. Furthermore, we apply mechanistic interpretability techniques to probe their internal computational states. Our analysis reveals that LLMs generally attempt to compute exact values or invoke external tools even in tasks that call for approximation. Moreover, while models sometimes reach the correct answer in early layers or steps, they still consume more tokens when solving approximation tasks. Additional experiments indicate that exact and approximate arithmetic operations rely on largely separate neural components. Drawing upon research on cognitive psychology, we argue that LLMs do not exhibit cognitive miserliness in the same way humans do in street math settings. We open source our work https://github.com/ctseng777/StreetMath
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。