用图像代替文字压缩思维链,让大模型更高效地推理
ImgCoT: Compressing Long Chain of Thought into Compact Visual Tokens for Efficient Reasoning of Large Language Model
- 将思维链转为图像作为压缩目标,避免语言形式干扰
- 新方法在多个数据集上用更少 token 达到更好推理效果
- 适合需要高效推理的场景,尤其对资源受限部署友好
将长链思维过程(CoT)压缩为紧凑的潜在标记对大语言模型(LLM)的高效推理至关重要。现有研究使用自编码器,以文本形式的思维链作为重建目标,导致潜在标记需保留表面语言特征(如词选择和语法),引入强烈的语言归纳偏置,弱化逻辑结构抽象能力。为此,我们提出ImgCoT,将重建目标从文本思维链替换为通过渲染生成的视觉思维链(visual CoT)。该方法以空间归纳偏置替代语言偏置,使潜在标记更关注推理步骤的空间布局,从而更好地捕捉全局推理结构。此外,尽管视觉潜在标记能编码抽象结构,但可能模糊细节。因此,我们设计松散版ImgCoT,通过低概率词元筛选关键文本推理步骤,与视觉潜在标记融合。此混合推理方式在保持全局结构的同时保留细粒度信息,且所需标记数远低于完整思维链。跨多个数据集和大模型的大量实验验证了两种版本ImgCoT的有效性。
原文摘要 · Abstract (English)
Compressing long chains of thought (CoT) into compact latent tokens is crucial for efficient reasoning with large language models (LLMs). Recent studies employ autoencoders to achieve this by reconstructing textual CoT from latent tokens, thus encoding CoT semantics. However, treating textual CoT as the reconstruction target forces latent tokens to preserve surface-level linguistic features (e.g., word choice and syntax), introducing a strong linguistic inductive bias that prioritizes linguistic form over reasoning structure and limits logical abstraction. Thus, we propose ImgCoT that replaces the reconstruction target from textual CoT to the visual CoT obtained by rendering CoT into images. This substitutes linguistic bias with spatial inductive bias, i.e., a tendency to model spatial layouts of the reasoning steps in visual CoT, enabling latent tokens to better capture global reasoning structure. Moreover, although visual latent tokens encode abstract reasoning structure, they may blur reasoning details. We thus propose a loose ImgCoT, a hybrid reasoning that augments visual latent tokens with a few key textual reasoning steps, selected based on low token log-likelihood. This design allows LLMs to retain both global reasoning structure and fine-grained reasoning details with fewer tokens than the complete CoT. Extensive experiments across multiple datasets and LLMs demonstrate the effectiveness of the two versions of ImgCoT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。