为大模型设计多层安全架构,防范提示注入攻击
Countermind: A Multi-Layered Security Architecture for Large Language Models
- 构建输入验证与语义边界,提前阻断恶意指令
- 动态限制模型内部语义路径,减少危险行为发生
- 支持多模态输入防护,适合高安全需求场景
大型语言模型应用的安全性面临“形式优先”攻击的严峻挑战,如提示注入和越狱攻击,恶意指令嵌入用户输入中。传统防御依赖输出后处理,脆弱且无法根治模型无法区分可信指令与不可信数据的问题。本文提出Countermind,一种多层安全架构,将防御从被动响应转向主动、预推理及推理中执行。该架构包括:(1) 语义边界逻辑(SBL)结合强制时间耦合文本加密器,降低明文提示注入攻击面;(2) 参数空间限制(PSR)机制,基于表示工程原理动态控制模型对内部语义簇的访问,缓解语义漂移与危险涌现行为;(3) 基于OODA环与学习型安全模块的可自调节核心,依据不可篡改审计日志动态更新防御策略;(4) 多模态输入沙箱与上下文防御机制,应对非文本数据威胁与长期语义污染。论文规划评估方案,量化该架构对形式优先攻击的攻击成功率(ASR)降低效果,并测量潜在延迟开销。
原文摘要 · Abstract (English)
The security of Large Language Model (LLM) applications is fundamentally challenged by "form-first" attacks like prompt injection and jailbreaking, where malicious instructions are embedded within user inputs. Conventional defenses, which rely on post hoc output filtering, are often brittle and fail to address the root cause: the model's inability to distinguish trusted instructions from untrusted data. This paper proposes Countermind, a multi-layered security architecture intended to shift defenses from a reactive, post hoc posture to a proactive, pre-inference, and intra-inference enforcement model. The architecture proposes a fortified perimeter designed to structurally validate and transform all inputs, and an internal governance mechanism intended to constrain the model's semantic processing pathways before an output is generated. The primary contributions of this work are conceptual designs for: (1) A Semantic Boundary Logic (SBL) with a mandatory, time-coupled Text Crypter intended to reduce the plaintext prompt injection attack surface, provided all ingestion paths are enforced. (2) A Parameter-Space Restriction (PSR) mechanism, leveraging principles from representation engineering, to dynamically control the LLM's access to internal semantic clusters, with the goal of mitigating semantic drift and dangerous emergent behaviors. (3) A Secure, Self-Regulating Core that uses an OODA loop and a learning security module to adapt its defenses based on an immutable audit log. (4) A Multimodal Input Sandbox and Context-Defense mechanisms to address threats from non-textual data and long-term semantic poisoning. This paper outlines an evaluation plan designed to quantify the proposed architecture's effectiveness in reducing the Attack Success Rate (ASR) for form-first attacks and to measure its potential latency overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。