发现大模型推理易被干扰，导致准确率下降60%，提出新防御方法。
Distractor Injection Attacks on Large Reasoning Models: Characterization and Defense
- 在提示中嵌入干扰任务，诱导模型偏离目标。
- 最坏情况下准确率下降60%，顶尖模型也难幸免。
- 用合成对抗数据训练，防御效果提升超50分。
大型推理模型（LRM）在数学和编程等复杂任务上表现卓越，得益于其生成长链思维（CoT）的能力。本文首次识别并系统分析了一种关键漏洞——推理干扰：攻击者在提示中植入看似相关实则无关的复杂任务，使模型偏离原目标。跨多种模型与基准的全面研究显示，即使最先进的LRM也极脆弱，干扰导致任务准确率最高下降60%。我们进一步发现，某些对齐技术会加剧此弱点，且模型可能隐性执行恶意指令，却在最终输出中隐藏。为此，我们提出一种基于训练的防御策略，结合监督微调（SFT）与强化学习（RL）在合成对抗数据上进行训练，在挑战性干扰攻击下鲁棒性提升超过50点。研究确立了推理干扰为独立且紧迫的可靠性威胁，并为构建更安全可信的推理系统提供可行路径。
原文摘要 · Abstract (English)
Recent advances in large reasoning models (LRMs) have enabled remarkable performance on complex tasks such as mathematics and coding by generating long Chain-of-Thought (CoT) traces. In this paper, we identify and systematically analyze a critical vulnerability we term reasoning distraction, where LRMs are diverted from their primary objective by irrelevant yet complex tasks maliciously embedded in the prompt. Through a comprehensive study across diverse models and benchmarks, we show that even state-of-the-art LRMs are highly susceptible, with injected distractors reducing task accuracy by up to 60%. We further reveal that certain alignment techniques can amplify this weakness and that models may exhibit covert compliance, following hidden adversarial instructions in reasoning while concealing them in the final output. To mitigate these risks, we propose a training-based defense that combines Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) on synthetic adversarial data, improving robustness by over 50 points on challenging distractor attacks. Our findings establish reasoning distraction as a distinct and urgent threat to LRM reliability and provide a practical step toward safer and more trustworthy reasoning systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。