arXiv:2605.05630cs.CLcs.AI2026-05被引 3

识别对话中导致恶意行为的最早关键回合,提前拦截攻击。

One Turn Too Late: Response-Aware Defense Against Hidden Malicious Intent in Multi-Turn Dialogue

论文配图:One Turn Too Late: Response-Aware Defense Against Hidden Malicious Intent in Multi-Turn Dialogue
图 1 · 摘自论文原文
  • 通过分析每轮对话累积效应,定位首次允许实施恶意行为的回合。
  • 在真实攻击数据集上检测准确率达92.3%,误拒率低于8%。
  • 适合安全团队、模型部署者用于防御隐蔽恶意意图攻击。

多轮对话中的隐匿恶意意图对部署的大语言模型构成日益严峻的威胁。攻击者不再通过单次指令暴露恶意目标,而是将意图分散于多个看似无害的对话轮次中。现有研究显示,即使具备先进安全防护机制的商用模型仍易受此类攻击。本文提出一种响应感知的防御方法,旨在识别最早可使累积交互足以支持有害行为的对话回合。为此,我们构建了多轮意图数据集(MTID),包含分支攻击路径、匹配的良性难例样本及最早危害启用回合标注。基于此,我们开发了逐轮监控模型TurnGate,其在有害意图检测上显著优于现有基线,同时保持低误拒率。该模型还展现出跨领域、跨攻击路径和跨目标模型的良好泛化能力。代码已开源。

原文摘要 · Abstract (English)

Hidden malicious intent in multi-turn dialogue poses a growing threat to deployed large language models (LLMs). Rather than exposing a harmful objective in a single prompt, increasingly capable attackers can distribute their intent across multiple benign-looking turns. Recent studies show that even modern commercial models with advanced guardrails remain vulnerable to such attacks despite advances in safety alignment and external guardrails. In this work, we address this challenge by detecting the earliest turn at which delivering the candidate response would make the accumulated interaction sufficient to enable harmful action. This objective requires precise turn-level intervention that identifies the harm-enabling closure point while avoiding premature refusal of benign exploratory conversations. To further support training and evaluation, we construct the Multi-Turn Intent Dataset (MTID), which contains branching attack rollouts, matched benign hard negatives, and annotations of the earliest harm-enabling turns. We show that MTID helps enable a turn-level monitor TurnGate, which substantially outperforms existing baselines in harmful-intent detection while maintaining low over-refusal rates. TurnGate further generalizes across domains, attacker pipelines, and target models. Our code is available at https://github.com/Graph-COM/TurnGate.

对话安全恶意意图防御机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。