发现大模型推理过程存在九类安全隐患,提出实时监控框架。
Beyond Content Safety: Real-Time Monitoring for Reasoning Vulnerabilities in Large Language Models
- 构建推理安全分类体系,识别九类不安全推理行为。
- 在4000+推理链上验证,87.11%步骤级定位准确率。
- 零样本外挂式监控,低延迟且抗对抗规避,适合部署使用。
大型语言模型越来越多地依赖显式的思维链推理来解决复杂任务,但推理过程本身的安全性尚未得到充分关注。现有工作主要聚焦内容安全(如检测有害、偏见或事实错误的输出),而将底层推理链视为不可见的中间产物。本文认为,推理安全是与内容安全并列的根本安全维度:要求模型的推理轨迹具备逻辑一致性、计算高效性,并能抵抗对抗性操纵。我们首次形式化推理安全概念,提出九类不安全推理行为的系统分类。通过大规模实证研究,在超过4000条推理链上(涵盖良性基准与四种前沿推理攻击)验证了所有九类错误均真实存在,且具有可解释的机制特征。为此,我们提出“推理安全监控器”——一种与目标LLM并行运行的外部零样本验证框架。它通过嵌入分类体系的提示实时检查每一步推理,一旦检测到不安全行为即触发中断信号。大量评估显示,该监控器在步骤级定位上达到最高87.11%的准确率,显著优于幻觉检测器和最佳过程奖励模型基线。关键优势在于对正确路径保持低误报率,延迟开销极小,并具备对自适应对抗规避的强鲁棒性。这些结果确立了推理安全监控在大型推理模型安全部署中的可行性与必要性。
原文摘要 · Abstract (English)
Large language models increasingly rely on explicit chain-of-thought reasoning to solve complex tasks, yet the safety of the reasoning process itself remains largely unaddressed. Existing work focuses predominantly on content safety (i.e., detecting harmful, biased, or factually incorrect outputs), while treating the underlying reasoning chain as an opaque intermediate artifact. We argue that reasoning safety constitutes a fundamental security dimension orthogonal to content safety: the requirement that a model's reasoning trajectory be logically consistent, computationally efficient, and resistant to adversarial manipulation. In this paper, we formalize reasoning safety and introduce a systematic taxonomy of nine unsafe reasoning behaviors. We then conduct a large-scale prevalence study, annotating over 4,000 reasoning chains across benign benchmarks and four state-of-the-art reasoning attacks, empirically demonstrating that all nine error types occur in practice with mechanistically interpretable signatures. To mitigate these threats, we propose the Reasoning Safety Monitor: an external, zero-shot verification framework that runs in parallel with the target LLM. It inspects each reasoning step in real time via a taxonomy-embedded prompt and dispatches an interrupt signal upon detecting unsafe behavior. Extensive evaluations show our monitor achieves up to 87.11% step-level localization accuracy, outperforming hallucination detectors and the best process reward model baselines by a substantial margin. Crucially, the monitor maintains a low false positive rate on correct reasoning paths, operates with negligible latency overhead, and exhibits robust resilience against adaptive adversarial evasion. These findings establish reasoning safety monitoring as a highly feasible and essential component for the secure deployment of large reasoning models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。