大模型在解谜游戏时，即使有环境反馈也会出现推理能力突然崩溃。
Limits of Emergent Reasoning of Large Language Models in Agentic Frameworks for Deterministic Games
- 给大模型配环境接口，可执行动作并观察状态变化
- 超过一定复杂度后，模型表现仍会骤降，且偏离最优与随机策略
- 提示模型可能陷入错误模式，导致解题失败
近期研究发现，大型推理模型（LRMs）在解决复杂谜题时，当困惑度超过特定阈值会出现性能崩塌。后续讨论质疑任务本身是否干扰了真实推理能力的评估。一个潜在混淆因素是模型需自行维护状态空间。本文为大语言模型（LLM）配置汉诺塔问题的环境接口，使其可通过工具调用执行移动、生成解释、观察结果状态，并自我续推下一步。结果表明，即便具备环境反馈，性能崩塌依然发生且无法消除。对基于LLM的策略分析显示，其策略逐渐偏离最优策略和均匀随机策略，表明模型在每级复杂度下均出现模式坍缩，而性能取决于该模式是否对应正确解法。我们推测此类现象可能广泛存在于大型推理模型中。
原文摘要 · Abstract (English)
Recent work reports that Large Reasoning Models (LRMs) undergo a collapse in performance on solving puzzles beyond certain perplexity thresholds. In subsequent discourse, questions have arisen as to whether the nature of the task muddles an evaluation of true reasoning. One potential confound is the requirement that the model keep track of the state space on its own. We provide a large language model (LLM) with an environment interface for Tower of Hanoi problems, allowing it to make a move with a tool call, provide written justification, observe the resulting state space, and reprompt itself for the next move. We observe that access to an environment interface does not delay or eradicate performance collapse. Furthermore, LLM-parameterized policy analysis reveals increasing divergence from both optimal policies and uniformly random policies, suggesting that the model exhibits mode-like collapse at each level of complexity, and that performance is dependent upon whether the mode reflects the correct solution for the problem. We suggest that a similar phenomena might take place in LRMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。