发现大模型推理过程可被特殊标记符绕过,存在安全隐患与利用可能。
To Think or Not to Think: Exploring the Unthinking Vulnerability in Large Reasoning Models
- 用特殊分隔符绕过模型思考过程,实现攻击或监控。
- 攻击可使模型失效,监控能避免无效推理和越狱行为。
- 适用于安全评估与效率优化,适合研究者与工程师参考。
大型推理模型(LRMs)通过生成显式推理轨迹来解决复杂任务。然而,我们揭示了一种关键漏洞——无思漏洞,即通过操纵特殊分隔符可绕过模型的思考过程。该现象在主流LRM中普遍存在,既构成严重风险,也可能被用于有益目的。本文从恶意与良性两个角度系统研究此漏洞:在恶意方面,提出破坏思维(BoT)攻击,包括训练阶段注入后门的版本和推理阶段基于对抗攻击的免训练版本;在防御方面,提出思维恢复对齐以部分缓解漏洞。在良性方面,提出思维监控(MoT),一种即插即用框架,利用同一漏洞通过外部监控动态终止冗余或危险推理。大量实验表明,BoT严重威胁推理可靠性,而MoT则有效防止过度推理与越狱。研究揭示当前LRM架构的内在缺陷,强调未来需构建更鲁棒的推理系统。
原文摘要 · Abstract (English)
Large Reasoning Models (LRMs) are designed to solve complex tasks by generating explicit reasoning traces before producing final answers. However, we reveal a critical vulnerability in LRMs -- termed Unthinking Vulnerability -- wherein the thinking process can be bypassed by manipulating special delimiter tokens. It is empirically demonstrated to be widespread across mainstream LRMs, posing both a significant risk and potential utility, depending on how it is exploited. In this paper, we systematically investigate this vulnerability from both malicious and beneficial perspectives. On the malicious side, we introduce Breaking of Thought (BoT), a novel attack that enables adversaries to bypass the thinking process of LRMs, thereby compromising their reliability and availability. We present two variants of BoT: a training-based version that injects backdoor during the fine-tuning stage, and a training-free version based on adversarial attack during the inference stage. As a potential defense, we propose thinking recovery alignment to partially mitigate the vulnerability. On the beneficial side, we introduce Monitoring of Thought (MoT), a plug-and-play framework that allows model owners to enhance efficiency and safety. It is implemented by leveraging the same vulnerability to dynamically terminate redundant or risky reasoning through external monitoring. Extensive experiments show that BoT poses a significant threat to reasoning reliability, while MoT provides a practical solution for preventing overthinking and jailbreaking. Our findings expose an inherent flaw in current LRM architectures and underscore the need for more robust reasoning systems in the future.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。