arXiv:2603.22871cs.AIcs.LG2026-03被引 1

用数学稳定机制让小模型高效解复杂逻辑题。

Dynamical Systems Theory Behind a Hierarchical Reasoning Model

  • 将递归推理转为连续微分方程,保证状态收敛到稳定点。
  • 500万参数模型在极难数独上达93.7%准确率,超大模型。
  • 压缩至26万参数仍保持85.4%性能,适合资源受限场景。

当前大语言模型依赖线性序列生成和海量参数,但在复杂算法推理上表现不佳。尽管近年提出的分层推理模型(HRM)和极小递归模型(TRM)证明紧凑递归网络可解决此类任务,但其训练动态缺乏严格数学保障,易导致不稳定和表征坍塌。本文提出收缩映射模型(CMM),将离散递归推理重构为连续神经常微分方程(NODEs)与随机微分方程(NSDEs)。通过显式约束隐状态收敛至稳定平衡态,并引入超球面排斥损失缓解特征坍塌,CMM实现了数学严谨且高度稳定的推理机制。在极难数独(Sudoku-Extreme)基准上,500万参数的CMM达到93.7%准确率,优于2700万参数的HRM(55.0%)和500万参数的TRM(87.4%)。更惊人的是,即使压缩至仅26万参数,CMM在Sudoku-Extreme上仍保持85.4%准确率,在迷宫任务上达82.2%。这确立了极致参数效率的新前沿,证明数学严谨的隐动态可有效替代盲目扩容来实现人工推理。

原文摘要 · Abstract (English)

Current large language models (LLMs) primarily rely on linear sequence generation and massive parameter counts, yet they severely struggle with complex algorithmic reasoning. While recent reasoning architectures, such as the Hierarchical Reasoning Model (HRM) and Tiny Recursive Model (TRM), demonstrate that compact recursive networks can tackle these tasks, their training dynamics often lack rigorous mathematical guarantees, leading to instability and representational collapse. We propose the Contraction Mapping Model (CMM), a novel architecture that reformulates discrete recursive reasoning into continuous Neural Ordinary and Stochastic Differential Equations (NODEs/NSDEs). By explicitly enforcing the convergence of the latent phase point to a stable equilibrium state and mitigating feature collapse with a hyperspherical repulsion loss, the CMM provides a mathematically grounded and highly stable reasoning engine. On the Sudoku-Extreme benchmark, a 5M-parameter CMM achieves a state-of-the-art accuracy of 93.7 %, outperforming the 27M-parameter HRM (55.0 %) and 5M-parameter TRM (87.4 %). Remarkably, even when aggressively compressed to an ultra-tiny footprint of just 0.26M parameters, the CMM retains robust predictive power, achieving 85.4 % on Sudoku-Extreme and 82.2 % on the Maze benchmark. These results establish a new frontier for extreme parameter efficiency, proving that mathematically rigorous latent dynamics can effectively replace brute-force scaling in artificial reasoning.

推理模型微分方程参数效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。