研究发现,越难懂的推理过程反而让大模型表现更好。
Do Cognitively Interpretable Reasoning Traces Improve LLM Performance?
- 用四种不同方式生成推理链,对比模型性能
- 真实推理链虽最差,但使模型准确率最高
- 适合关注模型效率而非可解释性的研究者
近期基于推理的大语言模型进展依赖于思维链(CoT)追踪,即模型在输出答案前生成中间推理步骤。这些追踪不仅用于引导推理,还作为小模型蒸馏的监督信号。一个普遍但常被忽视的假设是:这些推理链应语义清晰、对用户可理解。尽管有研究质疑其必要性,本文提出问题:‘思维链是否必须可解释才能提升模型性能?’我们在开放书问答任务中,通过监督微调LLaMA和Qwen模型,使用四种推理链类型:(1) DeepSeek R1的原始追踪,(2) LLM生成的R1追踪摘要,(3) LLM生成的后验解释,(4) 算法生成的可验证正确追踪。为量化可解释性与性能之间的权衡,我们进行了包含100名参与者的用户研究,评估各追踪类型的可理解性。结果表明存在显著矛盾:使用R1追踪微调的模型性能最强,但用户评分最低,认为其最不可理解。这说明中间推理步骤与最终用户可解释性可以解耦。
原文摘要 · Abstract (English)
Recent progress in reasoning-oriented Large Language Models (LLMs) has been driven by introducing Chain-of-Thought (CoT) traces, where models generate intermediate reasoning traces before producing an answer. These traces, as in DeepSeek R1, are not only used to guide inference but also serve as supervision signals for distillation into smaller models. A common but often implicit assumption is that CoT traces should be semantically meaningful and interpretable to the end user. While recent research questions the need for semantic nature of these traces, in this paper, we ask: ``\textit{Must CoT reasoning traces be interpretable to enhance LLM task performance?}" We investigate this question in the Open Book Question-Answering domain by supervised fine-tuning LLaMA and Qwen models on four types of reasoning traces: (1) DeepSeek R1 traces, (2) LLM-generated summaries of R1 traces, (3) LLM-generated post-hoc explanations of R1 traces, and (4) algorithmically generated verifiably correct traces. To quantify the trade-off between interpretability and performance, we further conduct a human-subject study with 100 participants rating the interpretability of each trace type. Our results reveal a striking mismatch: while fine-tuning on R1 traces yields the strongest performance, participants judged these traces to be the least interpretable. These findings suggest that it is useful to decouple intermediate tokens from end user interpretability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。