通过对抗性提示扰动,发现代码生成模型的推理能力极不稳定。
Chain-of-Code Collapse: Reasoning Failures in LLMs via Adversarial Prompting in Code Generation
- 设计语义忠实但结构对抗的提示扰动,测试模型推理鲁棒性。
- 部分扰动使准确率下降42.1%,个别反而提升35.3%。
- 揭示当前推理模型依赖表面模式,适合关注可信推理的研究者。
大型语言模型(LLMs)在需要复杂推理的任务中表现卓越,如代码生成、数学求解与算法合成——尤其在使用推理标记和思维链提示时。然而核心问题仍存:这些模型是否真正具备推理能力,还是仅依赖浅层统计模式?本文提出「链式代码坍塌」,系统研究推理型LLM的鲁棒性,引入一系列语义忠实但对抗性结构的提示扰动。评估覆盖700个源自LeetCode风格题目的扰动代码生成,采用故事重构、无关约束注入、示例重排与数值扰动等变换。结果表明,某些修改导致准确率下降最高达-42.1%,而其他情况却使准确率提升最高达+35.3%,显示模型不仅对语义敏感,也受表面提示动态影响。该发现暴露了当前推理系统的脆弱性与不可预测性,强调需发展更本质的推理对齐与提示鲁棒性方法。我们公开扰动数据集与评估框架,以推动可信、稳健的LLM推理研究。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have achieved remarkable success in tasks requiring complex reasoning, such as code generation, mathematical problem solving, and algorithmic synthesis -- especially when aided by reasoning tokens and Chain-of-Thought prompting. Yet, a core question remains: do these models truly reason, or do they merely exploit shallow statistical patterns? In this paper, we introduce Chain-of-Code Collapse, where we systematically investigate the robustness of reasoning LLMs by introducing a suite of semantically faithful yet adversarially structured prompt perturbations. Our evaluation -- spanning 700 perturbed code generations derived from LeetCode-style problems -- applies transformations such as storytelling reframing, irrelevant constraint injection, example reordering, and numeric perturbation. We observe that while certain modifications severely degrade performance (with accuracy drops up to -42.1%), others surprisingly improve model accuracy by up to 35.3%, suggesting sensitivity not only to semantics but also to surface-level prompt dynamics. These findings expose the fragility and unpredictability of current reasoning systems, underscoring the need for more principles approaches to reasoning alignments and prompting robustness. We release our perturbation datasets and evaluation framework to promote further research in trustworthy and resilient LLM reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。