提出时序感知防御框架,识别多轮对话中的隐蔽操纵行为
Temporal Context Awareness: A Defense Framework Against Multi-turn Manipulation Attacks on Large Language Models
- 通过分析语义漂移与跨轮意图一致性检测操纵
- 在模拟攻击中成功识别传统方法遗漏的细微操纵模式
- 适合关注对话系统安全的研究者与开发者
大型语言模型(LLMs)正面临复杂的多轮操纵攻击,攻击者通过看似无害的对话轮次逐步构建上下文以绕过安全机制,获取有害或未经授权的回复。这类攻击利用对话的时间特性,规避单轮检测方法,构成重大安全漏洞。本文提出时序上下文感知(Temporal Context Awareness, TCA)防御框架,通过动态上下文嵌入分析、跨轮一致性验证和渐进式风险评分,持续监测语义漂移、意图一致性和对话模式演变。初步评估显示,该框架能有效识别传统技术难以发现的细微操纵模式,在模拟攻击场景中展现出显著优势。研究还分析了多种攻击向量及其在多轮对话中的演进过程,揭示了对抗策略对模型漏洞的影响。结果强调了构建鲁棒、上下文感知防御机制的紧迫性,并将TCA视为保障LLM安全性的重要方向。代码已开源,支持该新兴安全领域的进一步研究。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly vulnerable to sophisticated multi-turn manipulation attacks, where adversaries strategically build context through seemingly benign conversational turns to circumvent safety measures and elicit harmful or unauthorized responses. These attacks exploit the temporal nature of dialogue to evade single-turn detection methods, representing a critical security vulnerability with significant implications for real-world deployments. This paper introduces the Temporal Context Awareness (TCA) framework, a novel defense mechanism designed to address this challenge by continuously analyzing semantic drift, cross-turn intention consistency and evolving conversational patterns. The TCA framework integrates dynamic context embedding analysis, cross-turn consistency verification, and progressive risk scoring to detect and mitigate manipulation attempts effectively. Preliminary evaluations on simulated adversarial scenarios demonstrate the framework's potential to identify subtle manipulation patterns often missed by traditional detection techniques, offering a much-needed layer of security for conversational AI systems. In addition to outlining the design of TCA , we analyze diverse attack vectors and their progression across multi-turn conversation, providing valuable insights into adversarial tactics and their impact on LLM vulnerabilities. Our findings underscore the pressing need for robust, context-aware defenses in conversational AI systems and highlight TCA framework as a promising direction for securing LLMs while preserving their utility in legitimate applications. We make our implementation available to support further research in this emerging area of AI security.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。