测试了多个Llama模型对大模型十大安全漏洞的防御能力，发现小模型反而更高效安全。
Benchmarking LLAMA Model Security Against OWASP Top 10 For LLM Applications
- 用对抗性提示测试不同Llama模型的安全检测能力。
- 小型专用模型Llama-Guard-3-1B检测率达76%，而大模型零检出。
- 开源了含标签的攻击数据集，支持可复现的AI安全研究。
随着大语言模型从研究原型转向企业系统，其安全漏洞对数据隐私与系统完整性构成严重威胁。本研究基于OWASP大模型十大应用安全风险框架，对多种Llama模型变体进行基准测试，评估威胁检测准确率、响应安全性及计算开销。在配备NVIDIA A30 GPU的FABRIC测试平台上，我们对五个标准Llama模型和五个Llama Guard变体，在覆盖十类漏洞的100个对抗性提示上进行了测试。结果显示安全性能差异显著：紧凑型Llama-Guard-3-1B模型检测率最高达76%，延迟仅0.165秒/次；而基础模型如Llama-3.1-8B检测准确率为0%，推理时间长达0.754秒。观察到模型规模与安全有效性呈反比关系，表明小型专用模型在安全任务中常优于大型通用模型。此外，我们公开了一个包含对抗性提示、威胁标签与攻击元数据的基准数据集，以支持可复现的AI安全研究。
原文摘要 · Abstract (English)
As large language models (LLMs) move from research prototypes to enterprise systems, their security vulnerabilities pose serious risks to data privacy and system integrity. This study benchmarks various Llama model variants against the OWASP Top 10 for LLM Applications framework, evaluating threat detection accuracy, response safety, and computational overhead. Using the FABRIC testbed with NVIDIA A30 GPUs, we tested five standard Llama models and five Llama Guard variants on 100 adversarial prompts covering ten vulnerability categories. Our results reveal significant differences in security performance: the compact Llama-Guard-3-1B model achieved the highest detection rate of 76% with minimal latency (0.165s per test), whereas base models such as Llama-3.1-8B failed to detect threats (0% accuracy) despite longer inference times (0.754s). We observe an inverse relationship between model size and security effectiveness, suggesting that smaller, specialized models often outperform larger general-purpose ones in security tasks. Additionally, we provide an open-source benchmark dataset including adversarial prompts, threat labels, and attack metadata to support reproducible research in AI security, [1].
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。