arXiv:2505.17066cs.CRcs.AI2025-05被引 4

用专家模型识别恶意提问,提升大模型在汽车领域的安全响应能力。

Improving LLM Outputs Against Jailbreak Attacks with Expert Model Integration

  • 引入轻量专家模型Archias分类用户问题类型
  • 结合分类结果生成防御性提示,显著降低越狱攻击成功率
  • 可适配多行业,适合企业级应用安全防护

在生产环境中使用大语言模型面临越狱攻击和提示注入等安全挑战,可能导致有害输出。尤其在特定领域(如汽车工业),通用模型可能无法处理专业问题。现有方法如领域微调和上下文学习仍不足以应对不断演化的攻击手段,且API模型灵活性不足。为此,我们提出Archias——一个小型专家模型,能有效区分领域内/外、恶意提问、价格注入、提示注入等问题。通过将Archias的分类结果整合进提示,引导主模型生成更安全的响应。该方法可灵活调整,适用于各类行业。我们构建了汽车行业的基准数据集,并开源以推动研究进展。

原文摘要 · Abstract (English)

Using LLMs in a production environment presents security challenges that include vulnerabilities to jailbreaks and prompt injections, which can result in harmful outputs for humans or the enterprise. The challenge is amplified when working within a specific domain, as topics generally accepted for LLMs to address may be irrelevant to that field. These problems can be mitigated, for example, by fine-tuning large language models with domain-specific and security-focused data. However, these alone are insufficient, as jailbreak techniques evolve. Additionally, API-accessed models do not offer the flexibility needed to tailor behavior to industry-specific objectives, and in-context learning is not always sufficient or reliable. In response to these challenges, we introduce Archias, an expert model adept at distinguishing between in-domain and out-of-domain communications. Archias classifies user inquiries into several categories: in-domain (specifically for the automotive industry), malicious questions, price injections, prompt injections, and out-of-domain examples. Our methodology integrates outputs from the expert model (Archias) into prompts, which are then processed by the LLM to generate responses. This method increases the model's ability to understand the user's intention and give appropriate answers. Archias can be adjusted, fine-tuned, and used for many different purposes due to its small size. Therefore, it can be easily customized to the needs of any industry. To validate our approach, we created a benchmark dataset for the automotive industry. Furthermore, in the interest of advancing research and development, we release our benchmark dataset to the community.

大模型安全越狱防御专家模型汽车行业

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。