小模型也能动态调整推理深度，更高效地处理复杂问题。
CosmicFish-HRM: Adaptive Reasoning via Hierarchical Recurrent Mechanisms in Compact Language Models
- 用分层推理模块动态决定每道题该算几轮
- 在不同任务上自动分配不同推理步数，表现更灵活
- 适合资源有限但需智能推理的场景
大语言模型虽具备强推理能力，但通常依赖海量参数和高计算成本。本文探索另一方向：在紧凑型语言模型中实现自适应推理深度。提出CosmicFish-HRM，其核心是分层推理模块（HRM），可在推理过程中动态分配计算资源。模型通过高低层次推理循环，根据输入复杂度自主决定何时停止，而非对所有输入进行固定计算。该模型融合现代Transformer组件，如分组查询注意力、旋转位置编码和SwiGLU激活函数。尽管推理结构在小规模下带来额外开销，但随着模型规模增大，其相对成本下降，优势逐渐显现。实验表明，模型能学习非均匀推理行为，在不同任务和输入间分配不同推理步数。结果说明，自适应推理深度或可成为替代单纯扩大参数量来提升推理能力的有效路径。
原文摘要 · Abstract (English)
Large language models have achieved strong reasoning capabilities, though often at the cost of massive parameter counts and expensive inference. In this work, we explore a different direction: adaptive reasoning depth in compact language models. We present CosmicFish-HRM, a compact language model built around a Hierarchical Reasoning Module (HRM) that dynamically allocates computational effort during inference. Instead of applying fixed computation to every input, the model iterates through high-level and low-level reasoning cycles and learns when to halt based on input complexity. CosmicFish-HRM combines this adaptive reasoning core with modern transformer components including Grouped Query Attention, RoPE, and SwiGLU activations. While the additional reasoning infrastructure introduces overhead at small scale, we hypothesize that this tradeoff becomes increasingly favorable as model size grows and the relative cost of the HRM core diminishes. Our results show that the model learns non-uniform reasoning behavior, allocating different numbers of reasoning steps across tasks and inputs. These findings suggest that adaptive reasoning depth may offer a promising alternative to relying solely on parameter scale for reasoning capability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。