arXiv:2607.20520cs.AIcs.HC2026-07

测试大模型解数学题时对表达形式的敏感度,发现换种说法答案就变错。

Representation Robustness Under Executable Reasoning Constraints in Large Language Models for Mathematical Problem Solving

  • 用不同表述方式(故事/符号/等价改写)测试同一数学题
  • 模型在等价问题间正确率波动大,改写后错误率上升明显
  • 代码验证虽暴露隐藏能力,但表达敏感仍存,需重新设计接口

大型语言模型(LLMs)在数学问题求解中日益受到关注,但以往研究常将语义等价的表达形式视为可互换,混淆了推理错误与接口失败。本文通过系统改变同一数学问题的表面表达形式——包括应用题、文字方程、符号方程及同构改写——评估五种主流LLM在直接生成答案条件下的表现。结果发现显著的表示敏感性:模型在等价表述间正确率波动剧烈,故事题、符号题与文字方程变体间存在非平凡的错误翻转率。此外,在同构重述下出现系统性性能下降,表明即使数学结构保持不变,细微的措辞变化也会导致性能退化。随后在代码增强条件下,模型将推理过程外化为可执行的Python代码并本地运行验证。该接口揭示了部分模型在直接提示下表现不佳却具备强潜在推理能力,但并未普遍提升鲁棒性;失败模式从模糊推理错误转向协议违规与执行失败。即使可执行推理成功,表示敏感性仍普遍存在。总体而言,推理支架无法消除表示脆弱性,而是暴露了正确性、可靠性、延迟与成本之间的新权衡。我们主张在大模型评估与部署中,将表示视为首要的接口设计变量,尤其在人工智能辅助求解系统中。

原文摘要 · Abstract (English)

Large language models (LLMs) are increasingly evaluated on mathematical problem solving, yet prior work often treats representationally equivalent formulations as interchangeable and conflates reasoning errors with interface failures. This paper investigates representation robustness in LLM-based mathematical problem solving by systematically varying surface representations of the same underlying problems, including story problems, word-equations, symbolic equations, and isomorphic paraphrases. Using a curated dataset of mathematically equivalent problems, we evaluate five contemporary LLMs under a direct answer generation condition. We find substantial representational sensitivity: models frequently change correctness across equivalent formulations, with nontrivial flip rates across story, symbolic, and word-equation variants. We also observe systematic regressions under isomorphic reformulations, showing that even subtle paraphrase-level changes can degrade performance despite preserved mathematical structure. We then evaluate a code-augmented condition in which models externalize reasoning as executable Python code that is run locally for validation. This interface reveals strong latent reasoning capability in some models that perform poorly under direct prompting, but it does not uniformly improve robustness. Instead, failures shift across interaction layers, from opaque reasoning errors to protocol violations and execution failures. Even when executable reasoning succeeds, representation sensitivity often persists. Overall, our results show that reasoning scaffolds do not eliminate representational brittleness, but expose new tradeoffs among correctness, reliability, latency, and cost. We argue that representation should be treated as a first-class interface design variable in LLM evaluation and deployment, especially for AI-assisted problem-solving systems.

数学推理表示鲁棒性代码增强接口设计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。