用压缩思维链提升模型推理效率，可动态调节计算量。
Compressed Chain of Thought: Efficient Reasoning Through Dense Representations
- 生成可变长度的连续思维符号，替代离散固定符号。
- 在多个任务上实现准确率提升，且计算量可控。
- 适合需要灵活推理与延迟权衡的应用场景。
思维链解码虽能提升语言模型的推理性能，但带来高生成延迟。现有方法使用离散嵌入集合中的固定长度序列作为思考令牌。本文提出压缩思维链（CCoT），通过生成内容丰富且连续的可变长度思考令牌，以密集表示显式推理链。该方法可适配现成的解码器语言模型。实验表明，通过额外的密集表示推理，准确率显著提升；同时，生成的思考令牌数量可动态控制，实现推理强度的按需调节。
原文摘要 · Abstract (English)
Chain-of-thought (CoT) decoding enables language models to improve reasoning performance at the cost of high generation latency in decoding. Recent proposals have explored variants of contemplation tokens, a term we introduce that refers to special tokens used during inference to allow for extra computation. Prior work has considered fixed-length sequences drawn from a discrete set of embeddings as contemplation tokens. Here we propose Compressed Chain-of-Thought (CCoT), a framework to generate contentful and continuous contemplation tokens of variable sequence length. The generated contemplation tokens are compressed representations of explicit reasoning chains, and our method can be applied to off-the-shelf decoder language models. Through experiments, we illustrate how CCoT enables additional reasoning over dense contentful representations to achieve corresponding improvements in accuracy. Moreover, the reasoning improvements can be adaptively modified on demand by controlling the number of contemplation tokens generated.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。