提出轻量级框架SafeDream,提前1-2轮检测大模型越狱攻击。
SafeDream: Safety World Model for Proactive Early Jailbreak Detection

- 用世界模型编码隐藏状态,预测安全风险演变趋势。
- 通过累积弱信号实现90%以上攻击提前1.06-1.20轮检测。
- 无需修改模型权重,适合部署于现有大模型系统。
多轮越狱攻击通过看似无害的对话逐步侵蚀大模型的安全对齐性,成功率超过90%。现有对齐方法存在三大缺陷:需昂贵权重修改、独立评估每轮对话、仅在生成有害内容后才检测。为此,我们首次提出主动早期检测问题,并引入检测提前量(detection lead)作为新指标,衡量攻击被发现前距模型响应的轮数。我们提出SafeDream框架,作为不修改模型权重的外部模块运行。其包含三部分:(1) 安全状态世界模型,将隐藏状态压缩为安全表征并预测跨轮演化;(2) CUSUM检测机制,累积每轮微弱风险信号以形成可靠证据;(3) 对比想象机制,在潜在空间同时推演攻击与正常未来路径,提前发出警报。在三个多轮越狱基准(XGuard-Train、SafeDialBench、SafeMTData)上,对比8个基线,SafeDream在所有数据集上均实现最佳检测及时性(1.06–1.20轮前),同时保持较低误报率,且整体检测质量更优。
原文摘要 · Abstract (English)
Multi-turn jailbreak attacks progressively erode LLM safety alignment across seemingly innocuous conversation turns, achieving success rates exceeding 90% against state-of-the-art models. Existing alignment-based and guardrail methods suffer from three key limitations: they require costly weight modification, evaluate each turn independently without modeling cumulative safety erosion, and detect attacks only after harmful content has been generated. To address these limitations, we first formulate the proactive early jailbreak detection problem with a new metric, detection lead, that measures how early an attack can be detected before the LLM complies. We then propose SAFEDREAM, a lightweight world-model-based framework that operates as an external module without modifying the LLM's weights. SAFEDREAM introduces three components: (1) a safety state world model that encodes LLM hidden states into a compact safety representation and predicts how it evolves across turns, (2) CUSUM detection that accumulates weak per-turn risk signals into reliable evidence, and (3) contrastive imagination that simultaneously rolls out attack and benign futures in latent space to issue early alarms before jailbreaks occur. On three multi-turn jailbreak benchmarks (XGuard-Train, SafeDialBench, SafeMTData) against 8 baselines, SAFEDREAM achieves the best detection timeliness across all benchmarks (1.06-1.20 turns before compliance) while maintaining competitive false positive rates and outperforming baselines in detection quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。