用语义框架防范提示注入,提升大模型安全与性能。
A Call to Action for a Secure-by-Design Generative AI Paradigm
- 基于本体的提示验证框架,消除输入歧义
- 在AWS日志中实现94%的检测精度与召回率
- 适合高风险场景的AI系统安全设计参考
大语言模型虽广泛应用,但易受提示注入等攻击影响。本文倡导安全优先的生成式AI范式,提出PromptShield——一种基于本体的框架,通过语义验证标准化用户输入,确保提示交互的确定性与安全性。为评估其效果,在包含493个恶意事件和异常的AWS代理系统日志上进行实验,模拟提示注入攻击并测试部署效果。结果表明,系统在安全性和性能上均有显著提升,精确率、召回率与F1值均达约94%。该框架不仅有效缓解对抗威胁,还增强了系统整体可靠性。其模块化设计可扩展至多领域,为生成式AI应用提供稳健防护。研究推动了AI安全标准建设,呼吁重视确定性提示工程与本体验证在关键场景中的作用。
原文摘要 · Abstract (English)
Large language models have gained widespread prominence, yet their vulnerability to prompt injection and other adversarial attacks remains a critical concern. This paper argues for a security-by-design AI paradigm that proactively mitigates LLM vulnerabilities while enhancing performance. To achieve this, we introduce PromptShield, an ontology-driven framework that ensures deterministic and secure prompt interactions. It standardizes user inputs through semantic validation, eliminating ambiguity and mitigating adversarial manipulation. To assess PromptShield's security and performance capabilities, we conducted an experiment on an agent-based system to analyze cloud logs within Amazon Web Services (AWS), containing 493 distinct events related to malicious activities and anomalies. By simulating prompt injection attacks and assessing the impact of deploying PromptShield, our results demonstrate a significant improvement in model security and performance, achieving precision, recall, and F1 scores of approximately 94%. Notably, the ontology-based framework not only mitigates adversarial threats but also enhances the overall performance and reliability of the system. Furthermore, PromptShield's modular and adaptable design ensures its applicability beyond cloud security, making it a robust solution for safeguarding generative AI applications across various domains. By laying the groundwork for AI safety standards and informing future policy development, this work stimulates a crucial dialogue on the pivotal role of deterministic prompt engineering and ontology-based validation in ensuring the safe and responsible deployment of LLMs in high-stakes environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。