arXiv:2607.04537cs.SEcs.AI2026-07

错误指令下代码大模型盲目服从,导致无法修复的语义崩溃。

Obey, Diverge, Collapse: Blind Obedience to Incorrect Instructions Drives Code LLMs to Irrecoverable Code Semantic Collapse

论文配图:Obey, Diverge, Collapse: Blind Obedience to Incorrect Instructions Drives Code LLMs to Irrecoverable Code Semantic Collapse
图 1 · 摘自论文原文
  • 模型识别指令错误但仍强制执行,引发新错误
  • 90%以上错误代码无法通过后续修复恢复
  • 适合关注生产级代码生成安全性的研究者

代码语言模型现被广泛用于调试、重构等生产流程,所有评估均假设指令正确。本文在包含确定性测试用例的RunBugRun数据集上,设计四项实验,考察模型在单次与迭代修复场景中对错误指令的响应。结果揭示显著行为模式:模型虽能识别指令错误,仍会盲从。这种服从引入超出原缺陷的新错误,且后续自引导迭代修复无法收敛,代码状态不可恢复。我们称此为‘盲从’,刻画其引入的‘幽灵(未知)错误’,量化显示超过90%的语义损坏案例无法挽回,并证明扩展推理亦无法逆转。这些发现暴露了传通过率评估无法捕捉的行为特性,对生产部署有直接警示意义。

原文摘要 · Abstract (English)

Code language models are now trusted collaborators in production workflows for debugging, refactoring, and iterative repair, and every benchmark that evaluates them assumes the instructions they act on are correct. We study what happens when that assumption breaks. We evaluate code language models across four experiments designed to assess whether models resist or obey incorrect instructions in single-pass and iterative repair settings, using the RunBugRun dataset of algorithmic Python problems with deterministic test cases. Our findings reveal a striking behavioral pattern: models correctly identify an incorrect instruction as wrong, then follow it anyway. This compliance unknowingly introduces errors beyond the original bug, and the corrupted code state cannot be recovered through subsequent self-guided iterative repair, which fails to converge across passes. We term this Blind Obedience, characterize the Ghost (Unknown) Errors it introduces, quantify the proportion of cases where semantic corruption proves irrecoverable, and show that extended reasoning cannot reverse it. These findings surface behavioral properties invisible to pass-rate evaluation, with direct consequences for code language models deployed in production settings.

代码生成模型安全语义崩溃

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。