用可解释性技术检测大模型漏洞，发现多个主流模型易被攻破。
Breaking Bad: Interpretability-Based Safety Audits of State-of-the-Art LLMs
- 基于可解释性方法，动态搜索触发不良行为的激活系数。
- Llama-3.3-70B-4bt模型在攻击下有高达91%响应被劫持。
- 该方法能系统化审计安全缺陷，适合模型开发者与安全研究者。
有效的大型语言模型（LLM）安全审计需要超越黑箱探测的工具，系统揭示模型内部漏洞。本文对八种前沿开源LLM（Llama-3.1-8B、Llama-3.3-70B-4bt、GPT-oss-20B、GPT-oss-120B、Qwen3-0.6B、Qwen3-32B、Phi4-3.8B、Phi4-14B）开展基于可解释性的越狱审计。利用通用引导（US）和表征工程（RepE）方法，提出自适应两阶段网格搜索算法，识别引发不安全行为的最佳激活引导系数。评估在精心筛选的有害查询集上进行，并采用标准化的LLM判断协议。结果表明，各模型鲁棒性差异显著：Llama-3系列高度脆弱，最大可被91%（US）和83%（RepE）的请求成功劫持；而GPT-oss-120B在两种方法下均保持稳健。Qwen与Phi系列表现混合，小模型（Qwen3-0.6B、Phi4-3.8B）劫持率较低，大模型则更易受攻击。研究证实可解释性引导是系统性安全审计的有效工具，但也揭示其双重用途风险，强调部署时需强化内部防御机制。
原文摘要 · Abstract (English)
Effective safety auditing of large language models (LLMs) demands tools that go beyond black-box probing and systematically uncover vulnerabilities rooted in model internals. We present a comprehensive, interpretability-driven jailbreaking audit of eight SOTA open-source LLMs: Llama-3.1-8B, Llama-3.3-70B-4bt, GPT-oss- 20B, GPT-oss-120B, Qwen3-0.6B, Qwen3-32B, Phi4-3.8B, and Phi4-14B. Leveraging interpretability-based approaches -- Universal Steering (US) and Representation Engineering (RepE) -- we introduce an adaptive two-stage grid search algorithm to identify optimal activation-steering coefficients for unsafe behavioral concepts. Our evaluation, conducted on a curated set of harmful queries and a standardized LLM-based judging protocol, reveals stark contrasts in model robustness. The Llama-3 models are highly vulnerable, with up to 91\% (US) and 83\% (RepE) jailbroken responses on Llama-3.3-70B-4bt, while GPT-oss-120B remains robust to attacks via both interpretability approaches. Qwen and Phi models show mixed results, with the smaller Qwen3-0.6B and Phi4-3.8B mostly exhibiting lower jailbreaking rates, while their larger counterparts are more susceptible. Our results establish interpretability-based steering as a powerful tool for systematic safety audits, but also highlight its dual-use risks and the need for better internal defenses in LLM deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。