通过语义图分析分离恶意提示,提升大模型安全防御能力
Disentangling Adversarial Prompts: A Semantic-Graph Defense for Robust LLM Security

- 用互信息分解提示语义,分离恶意与正常成分
- 在多个数据集上将有害输出降低85%以上
- 适合关注大模型安全的开发者和研究人员
大语言模型易受利用语义模糊性的对抗性提示攻击,导致有害或不当输出,威胁其在安全关键场景中的完整性与可用性。本文提出对抗性提示解耦(APD)框架,可主动识别并中和输入提示中的恶意成分。该框架融合三项创新:(1) 基于互信息的语义分解方法,实现对抗性与良性成分的统计独立分离;(2) 基于图谱分析的意图分类,检测提示语义中的恶意模式;(3) 在真实世界毒性与越狱提示数据集上训练的轻量级Transformer分类器,实现高效准确的对抗意图识别。在包含多种对抗性提示的数据集上评估显示,APD将有害输出生成减少超85%,同时对模型性能影响极小。该框架计算效率高,支持实时部署,为大模型安全提供可扩展、伦理合规的防御方案。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly vulnerable to adversarial prompts that exploit semantic ambiguities to bypass safety mechanisms, resulting in harmful or inappropriate outputs. Such attacks, including jailbreaking and prompt injection, pose significant risks to the integrity and availability of LLMs in security-critical applications. This paper proposes the Adversarial Prompt Disentanglement (APD) framework, a novel defense mechanism that proactively identifies and neutralizes malicious components in input prompts before they are processed by the LLM. The APD framework integrates three key innovations: (1) a mutual information-based semantic decomposition method to isolate adversarial and benign prompt components, ensuring statistical independence; (2) a graph-based intent classification approach that leverages spectral analysis to detect malicious patterns in prompt semantics; and (3) a lightweight transformer-based classifier trained on real-world datasets of toxic and jailbreaking prompts, enabling efficient and accurate adversarial intent detection. Evaluated on diverse datasets containing adversarial prompts, APD demonstrates superior robustness, reducing harmful output generation by over 85\% while maintaining negligible impact on model performance. The framework's computational efficiency supports real-time deployment, making it a practical solution for securing LLMs. Our work addresses critical challenges in machine learning security on novel attacks and integrity methods for ML systems, and offers a scalable, ethically grounded defense against prompt-based adversarial threats.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。