arXiv:2603.19558cs.CL2026-03

测试大模型推理对文本分类的提升效果,发现并非所有方法都有效且代价高昂。

TextReasoningBench: Does Reasoning Really Improve Text Classification in Large Language Models?

  • 设计新基准测试七种推理策略在文本分类中的表现
  • 复杂推理反而降低性能,小模型更易受损,增效仅1%-3%
  • 多数方法耗时耗资源十倍以上,性价比低,适合评估效率的场景

从大语言模型中提取显式、分步推理过程已成为提升模型能力的主流范式。尽管这类推理最初针对需要多步逻辑的任务设计,但如今被广泛应用于各类NLP任务。这一扩展隐含假设:推理对异构任务普遍有益。然而,其在文本分类任务中的实际效果,尤其在高昂的令牌与时间成本下,仍缺乏系统研究。为此,我们提出TextReasoningBench,一个用于评估大模型在文本分类中推理策略有效性与效率的系统性基准。我们在五项文本分类数据集上,对十种大模型使用七种推理策略(IO、CoT、SC-CoT、ToT、GoT、BoC和long-CoT)进行对比。除传统准确率与宏平均F1外,引入两项成本感知指标:每推理令牌带来的性能提升,以及性能提升相对于令牌开销增长的效率。实验发现:(1) 推理并非普遍提升分类表现;中等复杂度策略如CoT与SC-CoT带来稳定但有限的提升(大模型通常+1%至+3%),而更复杂的策略(如ToT、GoT)常无法超越基线,甚至导致性能下降,尤其在小模型上;(2) 推理往往效率低下:许多策略使令牌消耗增加10至100倍(如SC-CoT与ToT),但性能提升微弱。

原文摘要 · Abstract (English)

Eliciting explicit, step-by-step reasoning traces from large language models (LLMs) has emerged as a dominant paradigm for enhancing model capabilities. Although such reasoning strategies were originally designed for problems requiring explicit multi-step reasoning, they have increasingly been applied to a broad range of NLP tasks. This expansion implicitly assumes that deliberative reasoning uniformly benefits heterogeneous tasks. However, whether such reasoning mechanisms truly benefit classification tasks remains largely underexplored, especially considering their substantial token and time costs. To fill this gap, we introduce TextReasoningBench, a systematic benchmark designed to evaluate the effectiveness and efficiency of reasoning strategies for text classification with LLMs. We compare seven reasoning strategies, namely IO, CoT, SC-CoT, ToT, GoT, BoC, and long-CoT across ten LLMs on five text classification datasets. Beyond traditional metrics such as accuracy and macro-F1, we introduce two cost-aware evaluation metrics that quantify the performance gain per reasoning token and the efficiency of performance improvement relative to token cost growth. Experimental results reveal three notable findings: (1) Reasoning does not universally improve classification performance: while moderate strategies such as CoT and SC-CoT yield consistent but limited gains (typically +1% to +3% on big models), more complex methods (e.g., ToT and GoT) often fail to outperform simpler baselines and can even degrade performance, especially on small models; (2) Reasoning is often inefficient: many reasoning strategies increase token consumption by 10$\times$ to 100$\times$ (e.g., SC-CoT and ToT) while providing only marginal performance improvements.

大模型推理评估分类任务效率分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。