警告：别把模型中间输出当成人类思考过程
Position: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces!
- 指出中间生成标记被误称为'推理痕迹'是人为拟人化
- 强调这种说法误导对模型本质的理解和使用方式
- 呼吁研究者避免将技术输出与人类思维类比
中间令牌生成（ITG）已成为提升语言模型在推理任务上表现的标准方法，即模型在最终答案前生成一系列输出。这些中间输出常被称为‘推理痕迹’甚至‘思考痕迹’，隐含将其类比为人类解题时的思维步骤，从而为用户提供可解释的模型行为窗口。本文认为，这种拟人化并非无害比喻，而是危险误导——它混淆了模型的本质，影响有效使用，并导致可疑的研究实践。我们呼吁学术界停止对中间令牌的人格化描述。
原文摘要 · Abstract (English)
Intermediate token generation (ITG), where a model produces output before the solution, has become a standard method to improve the performance of language models on reasoning tasks. These intermediate tokens have been called \say{reasoning traces} or even \say{thinking traces} -- implicitly anthropomorphizing the traces, and implying that these traces resemble steps a human might take when solving a challenging problem, and as such can provide an interpretable window into the operation of the model's thinking process to the end user. In this position paper, we present evidence that this anthropomorphization isn't a harmless metaphor, and instead is quite dangerous -- it confuses the nature of these models and how to use them effectively, and leads to questionable research. We call on the community to avoid such anthropomorphization of intermediate tokens.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。