arXiv:2510.15211cs.LGcs.AI2025-10ACL被引 13

大模型推理时容易忽略指令,影响可靠性和安全性。

ReasonIF: Large Reasoning Models Fail to Follow Instructions During Reasoning

  • 构建新基准ReasonIF,评估模型在推理过程中的指令遵循能力。
  • 多数开源模型推理指令遵循率低于25%,难度越高越差。
  • 通过多轮推理和指令微调可提升遵循率,仍有巨大改进空间。

大型语言模型(LLMs)遵循用户指令的能力是其可靠性、安全性和实用性的重要基础。以往研究主要关注模型主响应中的指令遵循,而本文强调,大型推理模型(LRMs)在整个推理过程中也必须严格遵循指令。推理阶段的指令遵循能提升模型可控性与透明度,减少错误捷径、幻觉或奖励劫持的风险。为此,我们提出ReasonIF,一个系统化的推理指令遵循评估基准,涵盖六类指令,包括多语言推理、格式与长度控制等。在多个开源LRM(如GPT-OSS、Qwen3、DeepSeek-R1)上测试发现,推理指令遵循得分(IFS)最高不足0.25,即少于25%的推理轨迹符合指令要求。尤其当任务难度增加时,表现进一步下降。我们探索了两种提升策略:多轮推理和基于合成数据的推理指令微调(RIF)。RIF使GPT-OSS-20B的IFS从0.11提升至0.27,虽有进展,但仍远未达到理想水平。

原文摘要 · Abstract (English)

The ability of large language models (LLMs) to follow user instructions is central to their reliability, safety, and usefulness. While prior studies assess instruction adherence in the model's main responses, we argue that it is also critical for large reasoning models (LRMs) to follow user instructions throughout their reasoning process. Reasoning instruction following makes LRMs more controllable and transparent, while reducing risks of undesirable shortcuts, hallucinations, or reward hacking within reasoning traces. To evaluate this dimension, we introduce ReasonIF, a systematic benchmark for assessing reasoning instruction following. ReasonIF includes six categories of instruction prompts, spanning multilingual reasoning, formatting and length control. Across many open-source LRMs including GPT-OSS, Qwen3, and DeepSeek-R1, we find substantial failures in reasoning instruction adherence: the highest instruction following score (IFS) remains below 0.25, meaning that fewer than $25\%$ of reasoning traces comply with the given instructions. Notably, as task difficulty increases, reasoning instruction following degrades further. We also explore two strategies to enhance reasoning instruction fidelity. (1) multi-turn reasoning and (2) Reasoning Instruction Finetuning (RIF) using synthetic data. RIF improves the IFS of $GPT-OSS-20B$ from 0.11 to 0.27, indicating measurable progress but leaving ample room for improvement.

大模型推理指令遵循

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。