通过词元概率变化预测思维链收益，动态选择推理模式以减少计算量。
Token Signature: Predicting Chain-of-Thought Gains with Token Decoding Feature in Large Language Models
- 基于词元概率单调性设计两个评估指标，判断思维链是否有效。
- 动态思维链方法降低35%以上生成耗时，保持高准确率。
- 可迁移至闭源模型，适合追求效率的LLM应用部署。
思维链（CoT）技术在复杂推理任务中显著提升大语言模型性能，但其增益在不同任务间不一致，机制尚不明确。本文初步发现词元概率分布的单调性与CoT增益相关。据此提出两个基于概率分布的指标，评估不同任务中CoT的有效性。结合实例级指标与逻辑回归模型，提出动态思维链方法，按需选择使用CoT或直接作答。进一步将开源模型学习到的决策策略迁移到闭源模型。所提评估指标准确率达89.2%，动态思维链在保持高精度的同时，减少超过35%的词元消耗。本工作揭示了CoT作用机制的新视角，并提供了高效部署框架。
原文摘要 · Abstract (English)
Chain-of-Thought (CoT) technique has proven effective in improving the performance of large language models (LLMs) on complex reasoning tasks. However, the performance gains are inconsistent across different tasks, and the underlying mechanism remains a long-standing research question. In this work, we make a preliminary observation that the monotonicity of token probability distributions may be correlated with the gains achieved through CoT reasoning. Leveraging this insight, we propose two indicators based on the token probability distribution to assess CoT effectiveness across different tasks. By combining instance-level indicators with logistic regression model, we introduce Dynamic CoT, a method that dynamically select between CoT and direct answer. Furthermore, we extend Dynamic CoT to closed-source models by transferring decision strategies learned from open-source models. Our indicators for assessing CoT effectiveness achieve an accuracy of 89.2\%, and Dynamic CoT reduces token consumption by more than 35\% while maintaining high accuracy. Overall, our work offers a novel perspective on the underlying mechanisms of CoT reasoning and provides a framework for its more efficient deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。