用信息论量化推理步骤的增益，无标注也能精准发现大模型推理漏洞。
Understanding Chain-of-Thought in LLMs through Information Theory
- 通过信息增益衡量每步推理价值，不依赖人工标注数据。
- 在玩具算术、GSM8K和PRM800k上显著优于传统方法。
- 适合研究模型内部推理机制或提升评估准确性的研究人员。
大型语言模型（LLMs）通过链式思维（CoT）推理在复杂任务中表现出色，能将问题分解为可管理的子任务。然而，现有CoT评估方法要么需要标注的CoT数据，要么难以准确评估中间推理步骤，导致误报率较高。本文从信息论角度形式化了LLMs中的CoT推理，提出量化每个推理步骤信息增益的框架，无需昂贵的标注数据即可识别模型失败模式。我们在玩具算术、GSM8K和PRM800k数据集上进行了广泛实验，结果表明该方法显著优于基于结果的传统评估方法，在个体子任务上的性能洞察更为准确。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have shown impressive performance in complex reasoning tasks through the use of Chain-of-Thought (CoT) reasoning, allowing models to break down problems into manageable sub-tasks. However, existing CoT evaluation techniques either require annotated CoT data or fall short in accurately assessing intermediate reasoning steps, leading to high rates of false positives. In this paper, we formalize CoT reasoning in LLMs through an information-theoretic lens. Specifically, our framework quantifies the `information-gain' at each reasoning step, enabling the identification of failure modes in LLMs without the need for expensive annotated datasets. We demonstrate the efficacy of our approach through extensive experiments on toy arithmetic, GSM8K and PRM800k datasets, where it significantly outperforms existing outcome-based methods by providing more accurate insights into model performance on individual subtasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。