通过检测熵值突降现象,实时发现黑盒大模型的定向攻击。
DualSentinel: A Lightweight Framework for Detecting Targeted Attacks in Black-box LLM via Dual Entropy Lull Pattern
- 利用生成过程中的熵值异常稳定期识别攻击
- 检测准确率高,误报率近乎为零,开销极小
- 适合部署在实际应用中保护大模型安全
当前智能系统通过API集成强大大语言模型(LLMs),但其可信性可能受到后门攻击和提示注入攻击等定向攻击的严重威胁,这些攻击会秘密诱导模型生成特定恶意序列。现有防御方法通常依赖高权限访问、成本高昂且阻碍正常推理,难以应用于真实场景。为此,我们提出DualSentinel——一种轻量级统一防御框架,可实时准确检测目标攻击激活情况。我们首次发现受攻击模型具有‘熵值低谷’特征:当攻击成功劫持生成过程时,模型出现异常低且稳定的词元概率熵,表明其按固定路径输出而非自主决策。DualSentinel基于此设计双阶段检测机制:先通过感知幅度与趋势的监控方法实时敏感标记熵值低谷;一旦触发,则启动基于任务翻转的轻量级二次验证。仅当熵值低谷在原任务与翻转任务中均持续存在时,才确认攻击,证明输出受强制控制。大量实验表明,DualSentinel兼具极高有效性(检测准确率优异,近零误报)与显著高效性(额外开销可忽略),为部署中大模型的安全提供了切实可行方案。源代码可通过 https://doi.org/10.5281/zenodo.18479273 获取。
原文摘要 · Abstract (English)
Recent intelligent systems integrate powerful Large Language Models (LLMs) through APIs, but their trustworthiness may be critically undermined by targeted attacks like backdoor and prompt injection attacks, which secretly force LLMs to generate specific malicious sequences. Existing defensive approaches for such threats typically rely on high access rights, impose prohibitive costs, and hinder normal inference, rendering them impractical for real-world scenarios. To solve these limitations, we introduce DualSentinel, a lightweight and unified defense framework that can accurately and promptly detect the activation of targeted attacks alongside the LLM generation process. We first identify a characteristic of compromised LLMs, termed Entropy Lull: when a targeted attack successfully hijacks the generation process, the LLM exhibits a distinct period of abnormally low and stable token probability entropy, indicating it is following a fixed path rather than making creative choices. DualSentinel leverages this pattern by developing an innovative dual-check approach. It first employs a magnitude and trend-aware monitoring method to proactively and sensitively flag an entropy lull pattern at runtime. Upon such flagging, it triggers a lightweight yet powerful secondary verification based on task-flipping. An attack is confirmed only if the entropy lull pattern persists across both the original and the flipped task, proving that the LLM's output is coercively controlled. Extensive evaluations show that DualSentinel is both highly effective (superior detection accuracy with near-zero false positives) and remarkably efficient (negligible additional cost), offering a truly practical path toward securing deployed LLMs. The source code can be accessed at https://doi.org/10.5281/zenodo.18479273.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。