用隐式推理循环提升翻译质量,小模型也能达到大模型效果。
LatentMT: Machine Translation with Latent Reasoning

- 在隐藏状态中循环计算,不依赖参数量或显式思维链。
- 2.6B参数模型在32个语种对上媲美3-5倍大的模型。
- 推理效率高,适合资源少或算力有限的场景。
隐式推理循环语言模型(LoopLM)为机器翻译提供了不同于增大参数量或生成显式思维链的新路径:通过在隐藏状态内增加循环计算。我们提出首个系统性研究机器翻译中隐式推理循环模型的框架——LatentMT。该模型基于一个2.6B参数的小型主干网络,采用轻量级训练方法。在涵盖高、中、低资源语言的32个翻译方向上,其性能可媲美三到五倍大的模型,在高资源语言中表现具有竞争力,并在中低资源语言上达到当前最优水平。我们分析了循环推理步数的缩放行为,发现早期步骤中性能持续提升,随后迅速饱和。机制分析显示,隐藏表示差异随推理步数增加而缩小,支持性能饱和现象。此外,效率分析表明,LatentMT所需训练与推理计算量低于性能相当的大规模非隐式推理模型,证明隐式循环计算是实现紧凑、高效且强大的机器翻译的可行路径。
原文摘要 · Abstract (English)
Latent-reasoning looped language models (LoopLMs) offer a different scaling path for machine translation (MT): instead of increasing parameter count or emitting explicit chain-of-thought tokens, they spend additional recurrent computation inside hidden states. We introduce LatentMT, the first systematic study of latent-reasoning LoopLMs for machine translation. LatentMT adapts a small 2.6B-parameter backbone model with lightweight training. Across 32 translation directions spanning high-, mid-, and low-resource languages, LatentMT achieves performance comparable to models three to five times larger. It is competitive in a high-resource language and achieves state-of-the-art performance on both mid-resource and low-resource languages. Studying the behavior of scaling the number of recurrent reasoning steps, we find that recurrent computation consistently improves translation quality in early steps, then saturates quickly afterwards. Our mechanistic analysis shows that hidden-representation differences shrink along the recurrent reasoning-step axis, supporting the observed saturation in performance. Finally, our efficiency analysis shows that LatentMT requires lower training and inference compute than much larger non-latent-reasoning models with similar performance, making latent recurrent computation a promising path toward compact, efficient, and strong machine translation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。