让大模型在不生成中间步骤的情况下深度思考,提升推理效率与准确性。
Think Deep, Speak Once: Relit, A Recursive Latent Implicit Transformer Framework

- 用递归潜变量框架在隐藏状态中逐步优化推理过程
- 在GLoRE基准上以极少参数超越更大模型的逻辑推理表现
- 适合追求高效高质推理的AI研究者与工程应用
链式思维提示已成为激发大语言模型推理能力的主流范式,但其通过离散文本显式表达中间步骤,带来巨大计算开销。近年潜空间推理方法尝试将推理过程内化于连续隐藏状态中。其中,微型递归模型(TRMs)在符号推理上表现优异,但在自然语言场景中难以保持语义连贯性。为此,我们提出ReLIT(递归潜隐变压器),一种融合基础模型语义表征的混合框架。ReLIT在冻结的TinyLlama-1.1B主干上引入轻量级可训练递归模块,通过梯度隔离的循环结构迭代优化潜思(z),最终输出结果,实现从算法处理到语言直觉的结构化转变。实验表明,ReLIT在GLoRE逻辑推理基准上具备高参数效率,尽管监督极少,在ProofWriter和RuleTaker等挑战任务上达到或超过更大模型的表现。结果证明,可通过递归深度而非参数宽度有效扩展推理能力,为语义根基的隐式推理提供原则性框架。
原文摘要 · Abstract (English)
Chain-of-Thought (CoT) prompting has become the dominant paradigm for eliciting reasoning in Large Language Models (LLMs), yet it creates substantial computational overhead by forcing models to externalize intermediate reasoning steps as discrete tokens. Recent latent reasoning approaches attempt to internalize this process within continuous hidden states. One of the latest advancements in the field of latent reasoning, Tiny Recursive Models (TRMs) excel at symbolic reasoning but struggle to preserve semantic coherence in natural language settings. To bridge this gap, we introduce ReLIT (Recursive Latent Implicit Transformer), a hybrid framework that grounds deep recursive reasoning within the rich semantic representations of a foundational model. ReLIT augments a frozen LLM backbone (TinyLlama-1.1B) with a lightweight, trainable recursive block that iteratively refines its latent thinking (z) before committing to a final output, structurally solving linguistic intuition from algorithmic processing and enabling "deep thinking" via gradient-isolated recurrent loops without the latency of explicit token generation. Empirically, ReLIT achieves high parameter efficiency on the GLoRE logical reasoning benchmark, matching or outperforming significantly larger models on challenging tasks such as ProofWriter and RuleTaker despite minimal supervision. These results demonstrate that reasoning capability can be scaled efficiently through recurrent depth rather than parameter width, offering a principled framework for semantically grounded implicit reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。