只保留推理关键令牌，大幅降低大模型推理开销
Dynamic Thinking-Token Selection for Efficient Reasoning in Large Reasoning Models
- 通过注意力分析识别推理中的关键令牌
- 仅保留关键令牌的缓存，其余直接丢弃
- 适合追求高效推理的大模型应用
大型推理模型通过生成显式推理链来解决复杂问题，但这一过程带来显著的内存占用和计算开销。本文通过分析注意力图发现：推理链中仅有部分关键令牌对最终答案有决定性影响，其余贡献微乎其微。基于此，提出动态思考令牌选择（DynTS）方法，识别关键令牌并仅保留其对应的键值缓存状态，其余冗余缓存被清除，从而提升推理效率。
原文摘要 · Abstract (English)
Large Reasoning Models (LRMs) excel at solving complex problems by explicitly generating a reasoning trace before deriving the final answer. However, these extended generations incur substantial memory footprint and computational overhead, bottlenecking LRMs' efficiency. This work uses attention maps to analyze the influence of reasoning traces and uncover an interesting phenomenon: only some decision-critical tokens in a reasoning trace steer the model toward the final answer, while the remaining tokens contribute negligibly. Building on this observation, we propose Dynamic Thinking-Token Selection (DynTS). This method identifies decision-critical tokens and retains only their associated Key-Value (KV) cache states during inference, evicting the remaining redundant entries to optimize efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。