arXiv:2606.25524cs.AIcs.CL2026-06

发现大模型数学推理失败的致命单个触发词,可修复错误并提升准确率。

Cliff Tokens: Identifying Single-Token Failure Triggers in LLM Mathematical Reasoning

论文配图:Cliff Tokens: Identifying Single-Token Failure Triggers in LLM Mathematical Reasoning
图 1 · 摘自论文原文
  • 通过自适应阈值检测导致失败的精准触发词(悬崖词)。
  • 删除首个悬崖词后成功率恢复至1.0,保留则最高仅0.71。
  • 提出三类悬崖词分类,适用于优化模型推理能力。

大型语言模型在数学推理中表现优异,但相同问题下的不同推理路径结果差异显著:部分成功,部分失败。现有研究分析失败发生在步骤、片段或句子层面,或在失败已发生后的词上,均无法定位引发失败的精确触发词。本文提出‘悬崖词’概念——基于单侧两比例z检验,以随局部词级潜力自适应调整的阈值识别词级潜力骤降点。在七个模型和三个数学推理基准(GSM1K、MATH500、AIME 2025)上,悬崖词作为失败触发点:删除首个悬崖词并重采样后,pass@64恢复至1.0,而保留则恢复上限为0.71~1.00。进一步提出悬崖分类法:确定性、不确定性与采样偏移型悬崖,由贪婪选择与词熵定义。三类具有不同概率特征,且跨模型规模具泛化性。通过在悬崖位置进行单词偏好优化(Cliff-DPO),在GSM8K上训练后,各基准准确率最高提升+6.6;优化不确定与采样偏移型悬崖可改进推理,而确定性悬崖无效。

原文摘要 · Abstract (English)

Large language models (LLMs) reach high accuracy in mathematical reasoning, but individual traces on the same problem diverge; some arrive at the correct answer while others fail. Prior work analyzes failure at the step, chunk, or sentence level, or at tokens where failure has already occurred. Neither identifies the precise token that triggers the shift toward failure. We introduce the cliff token, a token where the token-wise potential drops significantly under an adaptive threshold that scales with the local token-wise potential, based on a one-sided two-proportion z-test. Across seven models and three mathematical reasoning benchmarks (GSM1K, MATH500, AIME 2025), cliff tokens act as failure triggers; deleting the first cliff token and resampling recovers pass@64 to 1.0, while keeping it limits recovery to between 0.71 and 1.00. We further introduce a cliff taxonomy of deterministic, uncertain, and sampled-off cliffs, defined by greedy choice and token entropy. Each type has distinct probabilistic characteristics, and the taxonomy generalizes across model scales. Finally, we validate the taxonomy via single-token preference optimization at cliff positions (Cliff-DPO). Trained on GSM8K, Cliff-DPO improves accuracy across benchmarks by up to +6.6. Optimizing at uncertain and sampled-off cliffs improves reasoning, while deterministic cliffs do not.

大模型数学推理故障分析悬崖词

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。