模型思考时容易被误导，辅助信息反而会放大错误。
Thinking in a Crowd: How Auxiliary Information Shapes LLM Reasoning
- 用科学问答数据集测试模型在干扰信息下的推理能力
- 误导信息使思考型模型准确率大幅下降，越思考越错
- 适合关注大模型可靠性与批判性推理的研究者
大型语言模型（LLMs）的推理能力是其在复杂知识密集型领域应用的核心。现实中，模型常依赖外部信息，这些信息可能有用、无关或具有误导性。本文通过基于ScienceQA构建的新数据集SciAux，系统研究了辅助信息对具备显式逐步推理能力的LLM推理过程的因果影响。结果揭示关键缺陷：模型的“思考模式”是一把双刃剑——有益信息可提升准确率，但误导信息会导致性能灾难性下降，且该负面影响随思考过程加剧。思考不仅未增强鲁棒性，反而强化了错误传播。这表明，挑战不在于让模型‘思考’，而在于赋予其评估推理依据信息的能力。SciAux数据集已公开于https://huggingface.co/datasets/billhdzhao/SciAux。
原文摘要 · Abstract (English)
The capacity of Large Language Models (LLMs) to reason is fundamental to their application in complex, knowledge-intensive domains. In real-world scenarios, LLMs are often augmented with external information that can be helpful, irrelevant, or even misleading. This paper investigates the causal impact of such auxiliary information on the reasoning process of LLMs with explicit step-by-step thinking capabilities. We introduce SciAux, a new dataset derived from ScienceQA, to systematically test the robustness of the model against these types of information. Our findings reveal a critical vulnerability: the model's deliberative "thinking mode" is a double-edged sword. While helpful context improves accuracy, misleading information causes a catastrophic drop in performance, which is amplified by the thinking process. Instead of conferring robustness, thinking reinforces the degree of error when provided with misinformation. This highlights that the challenge is not merely to make models "think", but to endow them with the critical faculty to evaluate the information upon which their reasoning is based. The SciAux dataset is available at https://huggingface.co/datasets/billhdzhao/SciAux.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。