用词汇索引坐标系实现思维状态的高效传递,提升大模型推理能力。
J-CoT: Chain-of-Thought in J-Space

- 提出J-space框架,用词汇索引系数表示中间推理状态
- 在数学、编程等任务上超越现有隐式推理方法,零样本表现达最优
- 无需生成完整语言中间步骤,适合追求效率的推理场景
思维链提示通过跨计算步骤传递中间状态来提升语言模型的推理能力。然而,仅依赖自然语言作为循环接口过于受限,因为许多临时计算无需完全语言化。现有隐式推理方法虽通过连续隐藏状态传播突破此限制,但会整体传递密集向量,缺乏对下一推理步骤所需信息的选择与组织机制。为此,本文提出基于J空间(词汇索引坐标系)的递归推理框架J-CoT。每个循环中模型在完整隐藏空间内计算;在循环边界,J-CoT将中间状态表达为词汇索引系数,以J-thought形式传递,并映射回隐藏表示进入下一轮。该方法既保持语言根基,又无需解码完整句子。在相同主干和推理设置下,J-CoT-Zero在所有基准测试中匹配或超越最强的隐式推理基线,而J-CoT-Train在数学、科学、编码及结构路径推理任务中取得最高分。
原文摘要 · Abstract (English)
Chain-of-thought prompting improves language-model reasoning by carrying intermediate states across successive computation steps. However, relying on natural language as the only recurrent interface is overly restrictive, since many transient computations do not need to be fully verbalized. Existing latent-reasoning methods remove this constraint by recurrently propagating continuous hidden states. However, these methods pass a dense hidden vector as a whole, without an explicit mechanism for selecting and organizing the information needed by the next reasoning step. This motivates an intermediate interface that remains linguistically grounded without requiring a decoded sentence. We introduce \textbf{J-CoT}, a recurrent reasoning framework built on \emph{J-space}, a vocabulary-indexed coordinate system within the model's hidden representations. Within each cycle, the model computes in its full hidden space. At the cycle boundary, J-CoT expresses the intermediate state as vocabulary-indexed coefficients, carries these coefficients forward as a \emph{J-thought}, and maps them back into the model's hidden representation for the next cycle. J-CoT therefore requires neither a fluent intermediate rationale nor recurrence over the complete hidden state. Under matched backbone and inference settings, J-CoT-Zero matches or exceeds the strongest evaluated latent-reasoning baseline on every benchmark, while J-CoT-Train obtains the highest score across the evaluated mathematical, scientific, coding, and structured path-reasoning tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。