用图像压缩推理过程，让大模型更快更省地处理长文本任务。
VTC-R1: Vision-Text Compression for Efficient Long-Context Reasoning
- 将中间推理步骤转为图像，通过视觉语言模型迭代记忆
- 实现3.4倍的令牌压缩率，端到端延迟降低2.7倍
- 适合需要高效长链推理的数学、逻辑类应用
长上下文推理显著提升了大语言模型处理复杂任务的能力，但带来了严重的效率瓶颈。现有高效方法常依赖复杂的额外训练或外部模型进行压缩，限制了可扩展性并丢失关键细粒度信息。本文提出VTC-R1，一种将视觉-文本压缩融入推理过程的新范式。不再直接处理冗长的文本推理轨迹，VTC-R1将中间推理片段渲染为紧凑图像，并将其作为“光学记忆”迭代输入视觉语言模型。基于OpenR1-Math-220K构建训练数据集，实现3.4倍的令牌压缩率，并微调代表性的视觉语言模型Glyph和Qwen3-VL。在MATH500、AIME25、AMC23和GPQA-D等基准上的大量实验表明，VTC-R1始终优于标准长上下文推理。此外，该方法显著提升推理效率，实现2.7倍的端到端延迟加速，展现出其在推理密集型应用中的可扩展潜力。代码已开源：https://github.com/w-yibo/VTC-R1。
原文摘要 · Abstract (English)
Long-context reasoning has significantly empowered large language models (LLMs) to tackle complex tasks, yet it introduces severe efficiency bottlenecks due to the computational complexity. Existing efficient approaches often rely on complex additional training or external models for compression, which limits scalability and discards critical fine-grained information. In this paper, we propose VTC-R1, a new efficient reasoning paradigm that integrates vision-text compression into the reasoning process. Instead of processing lengthy textual traces, VTC-R1 renders intermediate reasoning segments into compact images, which are iteratively fed back into vision-language models as "optical memory." We construct a training dataset based on OpenR1-Math-220K achieving 3.4x token compression and fine-tune representative VLMs-Glyph and Qwen3-VL. Extensive experiments on benchmarks such as MATH500, AIME25, AMC23 and GPQA-D demonstrate that VTC-R1 consistently outperforms standard long-context reasoning. Furthermore, our approach significantly improves inference efficiency, achieving 2.7x speedup in end-to-end latency, highlighting its potential as a scalable solution for reasoning-intensive applications. Our code is available at https://github.com/w-yibo/VTC-R1.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。