arXiv:2607.13899cs.AI2026-07中稿 · NeurIPS

测试大模型数学推理是否真会算,还是只靠捷径。

AIMO Interpretability Challenge

  • 基于数学奥赛题和模型内部机制,设计对抗性测试来区分真推理与假捷径。
  • 提供前沿模型访问权与新题目生成能力,评估其在复杂变体下的鲁棒性。
  • 适合关注AI可解释性、泛化能力与可靠性的研究者参与。

我们提出AIMO可解释性挑战,旨在通过模型内部机制区分前沿数学语言模型中稳健推理与虚假推理。该挑战源于标准推理评测的核心局限:高最终答案准确率无法揭示模型是依赖稳定推理机制,还是利用脆弱的推理捷径。基于人工智能数学奥林匹克(AIMO)题目及其符号表示,结合菲尔兹模型倡议资源,竞赛将提供:(1) 新发布的奥赛级数学推理题及其符号化表达,支持生成新型函数变体;(2) 前沿推理模型的访问权限;(3) 模型在这些问题上的对抗鲁棒性评估。参赛者将利用这些资源及计算基础设施,开发方法以识别哪些模型真正实现了稳健求解。本竞赛还将建立一个公开的鲁棒性基准与基线系统,为数学推理与可解释性领域提供持久的评测基础。科学层面,该挑战围绕人工智能研究的核心问题展开:能否判断,以及在多大程度上,前沿AI模型的决策具有泛化能力,从而具备可靠性?

原文摘要 · Abstract (English)

We propose the AIMO Interpretability Challenge, a competition on distinguishing robust from spurious reasoning in frontier mathematical language models based on the models' internal mechanisms. The challenge is motivated by a central limitation of standard reasoning benchmarks: strong final-answer accuracy does not reveal whether a model relies on stable reasoning mechanisms or exploits brittle reasoning shortcuts. Building on AI Mathematical Olympiad (AIMO) problems and submissions, together with resources from the Fields Model Initiative, the competition will provide (1) newly-published olympiad-level math reasoning problems and their symbolic representations, allowing generation of novel functional variants, (2) access to frontier reasoning models, and (3) assessments of models' adversarial robustness on these problems. Participants will use these resources, along with our computing infrastructure support, to develop methods for identifying which models solve problems robustly. Our competition will also create a new, open robustness benchmark and baseline systems, aiming to provide a lasting foundation for standard benchmarking in mathematical reasoning and interpretability. Scientifically, the competition connects interpretability and generalization research around a central question in AI research: can we determine if, and to what extent, the decision-making of frontier AI models is generalizable and thus, reliable?

可解释性数学推理鲁棒性AI评测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。