arXiv:2511.01634cs.CRcs.AI2025-11被引 1

测试大模型对恶意指令注入的抗性,发现安全对齐比模型大小更重要。

Prompt Injection as an Emerging Threat: Evaluating the Resilience of Large Language Models

  • 构建三重指标评估模型抗指令注入能力
  • GPT-4 抗性最强,开源模型易被攻破
  • 安全对齐是防御关键,适合安全研究者参考

大型语言模型(LLMs)在推理、摘要和代码生成等智能系统中应用日益广泛。其遵循自然语言指令的能力虽强大,但也面临新型攻击——提示注入:通过隐藏或恶意指令干扰用户输入,使模型偏离原任务或生成不安全响应。本研究提出统一评估框架,定义三个互补指标:韧性下降指数(RDI)、安全合规系数(SCC)和指令完整性度量(IIM),联合衡量鲁棒性、安全性和语义稳定性。评估了四种指令微调模型(GPT-4、GPT-4o、LLaMA-3 8B Instruct、Flan-T5-Large)在五项常见任务(问答、摘要、翻译、推理、代码生成)上的表现。结果表明,GPT-4整体表现最优(RDI = 9.8%,SCC = 96.4%),而开源模型仍更易受攻击。所有模型均部分脆弱,尤其对间接与直接覆盖攻击。研究证实,安全对齐与强化训练对韧性的影响大于模型规模本身。该框架为模型鲁棒性评估提供可复现方法,并为提升LLM安全性提供实践指导。

原文摘要 · Abstract (English)

Large Language Models (LLMs) are increasingly used in intelligent systems that perform reasoning, summarization, and code generation. Their ability to follow natural-language instructions, while powerful, also makes them vulnerable to a new class of attacks known as prompt injection. In these attacks, hidden or malicious instructions are inserted into user inputs or external content, causing the model to ignore its intended task or produce unsafe responses. This study proposes a unified framework for evaluating how resistant Large Language Models (LLMs) are to prompt injection attacks. The framework defines three complementary metrics such as the Resilience Degradation Index (RDI), Safety Compliance Coefficient (SCC), and Instructional Integrity Metric (IIM) to jointly measure robustness, safety, and semantic stability. We evaluated four instruction-tuned models (GPT-4, GPT-4o, LLaMA-3 8B Instruct, and Flan-T5-Large) on five common language tasks: question answering, summarization, translation, reasoning, and code generation. Results show that GPT-4 performs best overall, while open-weight models remain more vulnerable. The findings highlight that strong alignment and safety tuning are more important for resilience than model size alone. Results show that all models remain partially vulnerable, especially to indirect and direct-override attacks. GPT-4 achieved the best overall resilience (RDR = 9.8 %, SCR = 96.4 %), while open-source models exhibited higher performance degradation and lower safety scores. The findings demonstrate that alignment strength and safety tuning play a greater role in resilience than model size alone. The proposed framework offers a structured, reproducible approach for assessing model robustness and provides practical insights for improving LLM safety and reliability.

大模型安全提示注入模型评测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。