将推理与生成分离，提升大模型复杂推理的准确性与鲁棒性
JEPA-Reasoner: Decoupling Latent Reasoning from Token Generation
- 用联合嵌入预测架构实现纯潜在空间推理，独立于词元生成
- 0.9B模型在GSM8K上8跳任务准确率提升149.5%（相比耦合模型）
- 适合追求高可靠性推理系统的研究者和开发者
当前自回归语言模型将高层推理与底层词元生成耦合在一个序列过程中，导致推理轨迹易受表达误差累积影响。我们提出JEPA-Reasoner，一种新架构范式：使用联合嵌入预测架构（JEPA）进行纯潜在空间推理，同时配备独立的Talker模块完成语言重构。通过将推理引擎与离散词元采样过程解耦，该架构实现：(1) 错误隔离，词元级失败不会传播至潜在推理链；(2) 连续引导，生成器可获取无损推理轨迹；(3) 不确定性表示，通过混合潜在向量保留多个假设。在合成与自然语言任务上的控制实验表明，此解耦使0.9B模型在相同数据训练下，8跳GSM8K准确率相较耦合Transformer基线提升149.5%。本工作将重点从扩大耦合模型转向探索解耦架构，作为复杂推理更稳健的基础。
原文摘要 · Abstract (English)
Current autoregressive language models couple high-level reasoning and low-level token generation into a single sequential process, making the reasoning trajectory vulnerable to compounding expression errors. We propose JEPA-Reasoner, a novel architectural paradigm that decouples these tasks using a Joint-Embedding Predictive Architecture (JEPA) for pure latent-space reasoning and a separate Talker module for linguistic reconstruction. By isolating the reasoning engine from the discrete token-sampling process, our architecture enables: (1) Error Containment, where token-level failures cannot propagate into the latent reasoning chain; (2) Continuous Guidance, providing the generator with access to the entire lossless reasoning trajectory; and (3) Representation of Uncertainty, allowing the model to maintain multiple hypotheses via mixed latent vectors. Controlled experiments on synthetic and natural language tasks demonstrate that this decoupling enables a 0.9B model to achieve a 149.5\% improvement in 8-shot GSM8K accuracy over a coupled Transformer baseline trained on identical data. This work shifts the focus from scaling coupled models to investigating decoupled architectures as a more robust foundation for complex reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。