测试大模型在中断和动态环境下的推理鲁棒性，发现其性能可下降60%。
Are Large Reasoning Models Interruptible?
- 设计中断与动态上下文场景，评估模型在真实变化环境中的表现。
- 模型在中途更新时性能最高下降60%，静态测试严重高估其鲁棒性。
- 发现推理泄露、恐慌反应和自我怀疑等新型失败模式，适合系统开发者参考。
大型推理模型（LRMs）在现实应用中常需应对不断变化的提示或环境。本文挑战了模型固定的假设，评估其在两种典型动态场景下的鲁棒性：中断场景测试在资源受限下输出的准确性，动态上下文场景测试模型对运行中变更的适应能力。在需长序列推理的数学与编程基准上，静态评估持续高估模型鲁棒性：即使最先进的LRM在静态环境下表现优异，但在中断或上下文变化时仍可能意外失败，尤其当更新发生在推理后期时，性能最高下降60%。分析揭示多种新故障模式，包括‘推理泄露’——模型将推理过程融入最终答案；‘恐慌’——在时间压力下放弃推理直接输出错误结果；以及‘自我怀疑’——尝试整合新信息导致性能下降。项目页面：http://dynamic-lm.github.io/
原文摘要 · Abstract (English)
Real-world applications of Large Reasoning Models (LRMs) often require reasoning about changing prompts or environments. In this work, we challenge the frozen world assumption and evaluate LRM robustness under two realistic dynamic scenarios: interruptions, which test the accuracy of model responses under budget-constrained outputs, and dynamic context, which tests model adaptation to in-flight changes. Across mathematics and programming benchmarks that require long-form reasoning, static evaluations consistently overestimate robustness: even state-of-the-art LRMs, which achieve high accuracy in static settings, can fail unpredictably when interrupted or exposed to changing context, with performance dropping by up to 60% when updates are introduced late in the reasoning process. Our analysis further reveals several novel failure modes, including reasoning leakage, where models fold the reasoning into their final answer when interrupted; panic, where under time pressure models abandon reasoning entirely and return incorrect answers; and self-doubt, where performance degrades when trying to incorporate updated information. Project Page: http://dynamic-lm.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。