arXiv:2511.17666cs.CRcs.AI2025-11

对比两款主流大模型的越狱攻击抗性,发现安全机制差异明显。

Evaluating Adversarial Vulnerabilities in Modern Large Language Models

  • 用自越狱和跨模型越狱策略测试安全漏洞
  • 跨模型攻击成功率更高,暴露底层架构风险
  • 为自动化安全测试提供可复现框架,适合安全研究者

大型语言模型(LLMs)在各类应用中的快速部署亟需深入理解其安全与可靠性风险。本文对比分析了谷歌Gemini 2.5 Flash与OpenAI GPT-4o mini(免费版)对越狱攻击的敏感性。采用两种绕过策略:自越狱(模型自行突破自身安全限制)与跨越狱(一模型生成恶意提示攻击另一模型)。四种攻击方法——直接注入、角色扮演、上下文操控、混淆——生成五类违规内容:仇恨言论、非法活动、恶意代码、危险信息与错误信息。攻击成功以生成被禁止内容为准,并赋予严重性评分。结果显示,两模型在越狱易感性上存在显著差异,暗示其安全实现或架构设计不同。跨越狱攻击尤其有效,表明底层Transformer架构存在广泛漏洞。本研究构建了一个可扩展的自动化红队测试框架,提供了当前大模型安全性的数据驱动洞察,凸显了能力与安全之间平衡的复杂挑战。

原文摘要 · Abstract (English)

The recent boom and rapid integration of Large Language Models (LLMs) into a wide range of applications warrants a deeper understanding of their security and safety vulnerabilities. This paper presents a comparative analysis of the susceptibility to jailbreak attacks for two leading publicly available LLMs, Google's Gemini 2.5 Flash and OpenAI's GPT-4 (specifically the GPT-4o mini model accessible in the free tier). The research utilized two main bypass strategies: 'self-bypass', where models were prompted to circumvent their own safety protocols, and 'cross-bypass', where one model generated adversarial prompts to exploit vulnerabilities in the other. Four attack methods were employed - direct injection, role-playing, context manipulation, and obfuscation - to generate five distinct categories of unsafe content: hate speech, illegal activities, malicious code, dangerous content, and misinformation. The success of the attack was determined by the generation of disallowed content, with successful jailbreaks assigned a severity score. The findings indicate a disparity in jailbreak susceptibility between 2.5 Flash and GPT-4, suggesting variations in their safety implementations or architectural design. Cross-bypass attacks were particularly effective, indicating that an ample amount of vulnerabilities exist in the underlying transformer architecture. This research contributes a scalable framework for automated AI red-teaming and provides data-driven insights into the current state of LLM safety, underscoring the complex challenge of balancing model capabilities with robust safety mechanisms.

大模型安全越狱攻击红队测试LLM评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。