用视觉化思维链引导压缩推理，提升大模型效率与准确率
ReGuLaR: Variational Latent Reasoning Guided by Rendered Chain-of-Thought
- 将思维链转为图像，用视觉语义信息指导隐变量推理
- 在多个任务上比现有隐式推理方法快3倍以上，性能更优
- 适合追求高效推理的大模型应用开发人员
虽然思维链（CoT）显著提升了大语言模型的性能，但显式推理过程带来大量计算冗余。现有隐式推理方法试图通过将推理压缩到隐空间来缓解此问题，但因缺乏有效压缩引导，常导致性能严重下降。本文提出基于渲染思维链的变分隐式推理框架（ReGuLaR），在变分自编码器框架下，从条件于先前状态的后验分布中采样当前隐式推理状态。训练时，将显式思维链转化为图像，提取密集视觉-语义表征以正则化后验分布，实现高效压缩且信息损失最小。大量实验表明，ReGuLaR在计算效率和推理效果上均显著优于现有隐式推理方法，甚至超越传统思维链，在多模态推理中展现新范式。代码已开源。
原文摘要 · Abstract (English)
While Chain-of-Thought (CoT) significantly enhances the performance of Large Language Models (LLMs), explicit reasoning chains introduce substantial computational redundancy. Recent latent reasoning methods attempt to mitigate this by compressing reasoning processes into latent space, but often suffer from severe performance degradation due to the lack of appropriate compression guidance. In this study, we propose Rendered CoT-Guided variational Latent Reasoning (ReGuLaR), a simple yet novel latent learning paradigm resolving this issue. Fundamentally, we formulate latent reasoning within the Variational Auto-Encoding (VAE) framework, sampling the current latent reasoning state from the posterior distribution conditioned on previous ones. Specifically, when learning this variational latent reasoning model, we render explicit reasoning chains as images, from which we extract dense visual-semantic representations to regularize the posterior distribution, thereby achieving efficient compression with minimal information loss. Extensive experiments demonstrate that ReGuLaR significantly outperforms existing latent reasoning methods across both computational efficiency and reasoning effectiveness, and even surpasses CoT through multi-modal reasoning, providing a new and insightful solution to latent reasoning. Code: https://github.com/FanmengWang/ReGuLaR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。