指令遵循反而降低大模型解题能力，研究发现并量化了这一反直觉现象。
On the Paradoxical Interference between Instruction-Following and Task Solving
- 提出SUSTAINSCORE指标，通过添加自明约束评估指令对任务的干扰
- 数学、多跳问答、代码生成任务中性能显著下降，连Claude-Sonnet-4.5也受影响
- 干扰源于模型过度关注指令，适合关注对齐机制与模型行为的研究者
指令遵循旨在通过明确任务执行方式来对齐大语言模型与人类意图。然而我们揭示了一个反直觉现象：指令遵循可能反而干扰模型的任务解决能力。为此我们提出了SUSTAINSCORE指标，用于量化指令遵循对任务解决的干扰程度。该指标通过在指令中插入一个原本由成功输出自然满足的自明约束，测量任务性能下降情况。在当前主流大模型上进行的数学、多跳问答和代码生成实验表明，加入此类自明约束后性能大幅下降，甚至影响Claude-Sonnet-4.5等先进模型。我们验证了干扰现象在不同约束类型和模型规模下的普遍性，并识别出常见失败模式。进一步分析发现，失败案例相比成功案例显著分配更多注意力给指令约束。最后，利用SUSTAINSCORE初步探究了不同后训练范式对干扰的影响，为当前对齐策略提供了实证观察。代码与数据将公开，以促进后续研究。
原文摘要 · Abstract (English)
Instruction following aims to align Large Language Models (LLMs) with human intent by specifying explicit constraints on how tasks should be performed. However, we reveal a counterintuitive phenomenon: instruction following can paradoxically interfere with LLMs' task-solving capability. We propose a metric, SUSTAINSCORE, to quantify the interference of instruction following with task solving. It measures task performance drop after inserting into the instruction a self-evident constraint, which is naturally met by the original successful model output and extracted from it. Experiments on current LLMs in mathematics, multi-hop QA, and code generation show that adding the self-evident constraints leads to substantial performance drops, even for advanced models such as Claude-Sonnet-4.5. We validate the generality of the interference across constraint types and scales. Furthermore, we identify common failure patterns, and by investigating the mechanisms of interference, we observe that failed cases allocate significantly more attention to constraints compared to successful ones. Finally, we use SUSTAINSCORE to conduct an initial investigation into how distinct post-training paradigms affect the interference, presenting empirical observations on current alignment strategies. We will release our code and data to facilitate further research
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。