让压缩推理过程自动生成可读解释,兼顾效率与透明度。
Think in Latent, Explain in Language: Self-Explainable Latent Reasoning

- 用双目标训练让模型同时优化答案准确性和自我解释能力。
- 在多个模型上实现更高精度和更低显存消耗,解释无需额外模块。
- 适合需要可解释性的高效率推理场景,如医疗、金融决策系统。
隐式推理通过将冗长的思维过程压缩为紧凑嵌入,显著提升了计算效率,但导致推理过程不透明。现有方法面临两难:要么是不可解释的黑箱(如Coconut),要么依赖独立后处理解码器(如Heima),带来架构开销且解释与推理分离。本文提出统一的自解释隐式推理框架SELR,通过联合优化两个目标:(1)答案损失,确保隐式推理轨迹输出正确答案;(2)思维链损失,训练模型将自身隐式表示还原为人类可读的推理步骤。该设计使隐式表示既高效又具备语义可解释性,无需外部解码器。我们在大语言模型和视觉-语言模型上验证了SELR的有效性,结果表明其相比基线在保持更高准确率的同时,实现了更优的令牌效率,并首次实现无需辅助模型的自包含可解释性。
原文摘要 · Abstract (English)
Latent reasoning has emerged as a powerful alternative to text-based Chain-of-Thought (CoT), offering significant gains in computational efficiency by compressing verbose reasoning into compact embeddings. However, compressing reasoning into the latent space renders the thinking opaque, hindering its interpretability. Current methods present a stark trade-off: they either function as unexplainable ''black boxes'' (e.g., Coconut), where the latent reasoning is not human-readable, or rely on separate post-hoc decoders for explainability (e.g., Heima), introducing architectural overhead and decoupling the explanation from the actual reasoning process. In this work, we present a unified framework for Self-Explainable Latent Reasoning (SELR) that trains a single model to perform efficient and inherently explainable latent reasoning. Our core contribution is a novel multi-task training objective that optimizes for two goals simultaneously: (1) an Answer Loss that optimizes the latent reasoning trajectory to produce accurate final answers, and (2) a CoT Loss that explicitly trains the same model to decode its own latent representations back into human-understandable reasoning steps. This design ensures that generated latent representations are both task-effective and semantically interpretable, eliminating the need for external decoders. We validate the effectiveness of SELR on both Large Language Models (LLMs) and Vision-Language Models (VLMs), demonstrating that SELR achieves superior token efficiency and accuracy compared to baselines, while uniquely providing self-contained explainability without auxiliary models. Project page is available at https://jasondayuan.github.io/SELR/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。