通过分步分解任务，可显著降低大模型分类错误率。
How does Chain of Thought decompose complex tasks?
- 将复杂任务拆分为多层小分类问题，模拟思维链结构。
- 当分类类别数低于临界值时，深度思考反而增加错误率。
- 存在最优思维深度，超过后无法进一步提升性能。
许多语言任务可建模为分类问题：大语言模型（LLM）接收提示后从多个可能答案中选择一个。我们发现此类问题的分类误差随类别数量呈幂律增长。这带来重大影响：通过将整体任务分解为一系列具有相同类别数（‘度’）的小分类问题，可显著降低预测误差。这种树状分解结构即链式思维（CoT）。已有观察表明，当模型‘思考’更深入（即构建更深的树）时，基于CoT的预测器表现更好。我们识别出一个关键阈值——当度低于此值时，思考反而有害；高于此值时，存在一个最优深度使误差最小。进一步增加思考深度无法突破该最小误差。
原文摘要 · Abstract (English)
Many language tasks can be modeled as classification problems where a large language model (LLM) is given a prompt and selects one among many possible answers. We show that the classification error in such problems scales as a power law in the number of classes. This has a dramatic consequence: the prediction error can be reduced substantially by splitting the overall task into a sequence of smaller classification problems, each with the same number of classes ("degree"). This tree-structured decomposition models chain-of-thought (CoT). It has been observed that CoT-based predictors perform better when they "think", i.e., when they develop a deeper tree, thus decomposing the problem into a larger number of steps. We identify a critical threshold for the degree, below which thinking is detrimental, and above which there exists an optimal depth that minimizes the error. It is impossible to surpass this minimal error by increasing the depth of thinking.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。