提出在变压器隐空间中进行分层推理，提升逻辑任务表现。
Hierarchical Reasoning Models: Perspectives and Misconceptions
- 在模型隐空间引入循环推理机制，突破传统顺序预测限制。
- 在多种二维推理任务上实现显著性能提升，验证方法有效性。
- 澄清常见误解，为后续研究提供设计参考，适合逻辑推理方向研究者。
Transformer 在自然语言处理等任务中表现出色，主要依赖于序列化的自回归下一个词预测。然而，在逻辑推理任务上表现不佳，未必源于模型根本缺陷，而可能是因为未充分探索更创新的使用方式，如隐空间和循环推理。近年来，一种名为分层推理模型（Wang et al., 2025）的新方法在变压器的隐空间中引入了新型循环推理机制，在多项二维推理任务上取得显著成果。尽管前景广阔，该类模型仍处于初期阶段，亟需深入研究。本文综述此类模型，分析关键设计选择，测试替代变体，并澄清常见误解。
原文摘要 · Abstract (English)
Transformers have demonstrated remarkable performance in natural language processing and related domains, as they largely focus on sequential, autoregressive next-token prediction tasks. Yet, they struggle in logical reasoning, not necessarily because of a fundamental limitation of these models, but possibly due to the lack of exploration of more creative uses, such as latent space and recurrent reasoning. An emerging exploration in this direction is the Hierarchical Reasoning Model (Wang et. al., 2025), which introduces a novel type of recurrent reasoning in the latent space of transformers, achieving remarkable performance on a wide range of 2D reasoning tasks. Despite the promising results, this line of models is still at an early stage and calls for in-depth investigation. In this work, we review this class of models, examine key design choices, test alternative variants and clarify common misconceptions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。