arXiv:2606.16527cs.CRcs.CL2026-06

通过结构验证与语义审计,有效识别并防御黑盒越狱攻击。

DoubtProbe: Black-Box Jailbreak Defense via Structural Verification and Semantic Auditing

论文配图:DoubtProbe: Black-Box Jailbreak Defense via Structural Verification and Semantic Auditing
图 1 · 摘自论文原文
  • 双分支架构:分析请求结构一致性与语义安全性
  • 在Qwen2.5-72B上将越狱攻击成功率降至0.100(原0.293)
  • 适用于大模型部署场景,对新型攻击具强鲁棒性

随着大语言模型在用户系统中的广泛应用,黑盒越狱防御成为重要实际问题。现有方法依赖已知攻击覆盖、提示级语义判断或本地运行时控制,但在提示包装、表达重写和结构操纵下易失效。我们观察到,许多黑盒越狱并未消除有害目标,而是重组表达与执行所需信息,从而规避安全对齐但仍可生成恢复。基于此,提出DoubtProbe,一种结合结构验证与语义审计的双分支推理时防御框架,将黑盒越狱防御建模为受控变换下的一致性检查。结构分支从原始请求提取结构化表示,约束重构后检测原与重构请求间的信息保留失败;语义分支直接审计原始提示。在代表性黑盒防御上评估了其对越狱与良性请求的性能,并测试了从Qwen2.5-72B到Llama-3.1-70B的骨干迁移。结果表明,DoubtProbe实现了更强且更稳定的防御-效用权衡:在Qwen2.5-72B上,将JBB攻击成功率从0.293降至0.100,CodeAttack从0.152降至0.001,同时在AlpacaEval和OR-Bench上的误报率分别保持在0.022和0.016;该模式在Llama-3.1-70B上仍稳定。研究显示,结构不一致信号为黑盒越狱防御提供了实用且可泛化的基础,尤其与语义审计结合时优势显著。

原文摘要 · Abstract (English)

As large language models (LLMs) are increasingly deployed in user-facing systems, black-box jailbreak defense has become an important practical problem. Existing defenses often rely on known-attack coverage, prompt-level semantic judgment, or local runtime control, yet these paths can become unstable under evolving prompt packaging, expression rewriting, and structure manipulation. We observe that many black-box jailbreaks do not remove the harmful goal, but reorganize the information needed to express and execute it, thereby evading safety alignment while remaining recoverable during generation. Motivated by this observation, we propose DoubtProbe, a dual-branch inference-time defense framework that combines structural verification with semantic auditing and formulates black-box jailbreak defense as consistency checking under controlled transformation. The structural branch extracts a structured representation from the original request, reconstructs the request under representation constraints, and detects information-preservation failures between the original and reconstructed requests; the semantic branch audits the original prompt directly. We evaluate DoubtProbe against representative black-box defenses on jailbreak and benign-request benchmarks, and further test backbone transfer from Qwen2.5-72B to Llama-3.1-70B. Results show that DoubtProbe achieves a stronger and more stable defense-utility trade-off: on Qwen2.5-72B, it reduces the JBB attack success rate from 0.293 to 0.100 and the CodeAttack attack success rate from 0.152 to 0.001, while maintaining false positive rates of 0.022 and 0.016 on AlpacaEval and OR-Bench; the same pattern remains stable on Llama-3.1-70B. These findings show that structural inconsistency signals provide a practical and generalizable basis for black-box jailbreak defense, especially when combined with semantic auditing.

模型安全越狱防御结构验证语义审计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。