将思维链建模为可回溯的隐状态计算，提升推理泛化能力。
Reproducing Recurrent Transformers: The CoTFormer
- 用可回溯的隐状态模拟思维链推理过程。
- 在推理任务上显著提升分布外泛化性能。
- 适合研究大模型推理机制与可解释性的人参考。
CoTFormer 架构将思维链（Chain-of-Thought）形式化为一种递归隐状态计算，保留中间状态作为可关注的表示，以模仿显式的推理轨迹。本文在困惑度和计算效率指标上评估了 CoTFormer 及其结构变体。此外，我们扩展评估至受控算法场景，以检验该递归框架是否能在归纳推理任务中提升分布外泛化能力。
原文摘要 · Abstract (English)
The CoTFormer architecture formalizes Chain-of-Thought as a form of recurrent latent computation, preserving intermediate states as attendable representations to mimic explicit reasoning traces. In this work, we evaluate CoTFormer and its structural variants across perplexity and compute efficiency metrics. Furthermore, we extend evaluation to controlled algorithmic settings to determine whether this recurrent framework improves out-of-distribution generalisation on inductive reasoning tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。