统一压缩与并行生成，让大模型推理更快更省显存。
MemoSight: Unifying Context Compression and Multi Token Prediction for Reasoning Acceleration
- 用特殊标记和位置布局统一设计压缩与并行预测
- 显存占用降66%，推理速度提56%，准确率损失<3%
- 适合追求高效推理的复杂任务部署场景
链式思维（CoT）虽能提升大模型推理能力，但其线性增长的键值缓存（KV cache）带来巨大内存与推理开销。现有方法通过上下文压缩和多标记并行生成分别优化效率，但因训练范式与架构假设不同，难以有效融合。本文提出MemoSight（基于记忆-远见的推理框架），采用共享极简设计，利用特殊标记与标记专属位置布局，统一实现上下文压缩与多标记并行预测。在四个推理基准上的实验表明，相比原始监督微调基线，MemoSight将KV缓存使用减少高达66%，推理速度提升56%，平均推理准确率下降不足3%，优于现有CoT压缩方法的效率-精度平衡。
原文摘要 · Abstract (English)
While chain-of-thought (CoT) reasoning enables LLMs to solve challenging reasoning tasks, the linear growth of the KV cache leads to substantial memory and inference overhead. Existing approaches such as context compression and multi-token prediction (MTP) improve efficiency from two complementary directions by compressing historical tokens and generating future tokens in parallel. However, effectively combining them remains challenging due to their different training paradigms and architectural assumptions. In this work, we propose MemoSight (Memory-Foresight-Based Reasoning), a unified framework that integrates context compression and MTP to improve inference efficiency while preserving CoT performance. MemoSight adopts a shared minimalist design based on special tokens and token-specific positional layouts for both compression and parallel prediction. Experiments on four reasoning benchmarks show that, compared to the vanilla SFT baseline, MemoSight reduces KV cache usage by up to 66% and improves inference speed by 56%, while incurring less than a 3% drop in average reasoning accuracy, yielding a better efficiency-accuracy trade-off than existing CoT compression methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。