用提问方式零样本拦截多模态有害提示，提升大模型安全防护能力。
QGuard:Question-based Zero-shot Guard for Multi-modal LLM Safety
- 通过设计多样化提问来检测并拦截有害输入，无需微调。
- 在文本与多模态有害数据集上均表现良好，有效防御新型攻击。
- 支持白盒分析用户输入，适合实际部署中的安全防护场景。
大型语言模型（LLMs）的快速发展对多个领域产生了深远影响，但同时也增加了恶意用户利用有害或越狱提示进行攻击的风险。尽管已有多种方法尝试阻止此类提示，但防范恶意攻击仍是重要且具有挑战性的任务。本文提出QGuard，一种简单而有效的零样本安全防护机制，通过问题提示的方式，在不依赖微调的情况下阻断文本及多模态有害提示。通过多样化和修改防护问题，该方法对最新有害提示仍具鲁棒性。实验表明，QGuard在纯文本与多模态有害数据集上均表现出色。此外，通过对问题提示的分析，实现了对用户输入的白盒洞察。本方法为实际部署中降低有害提示带来的安全风险提供了重要参考。
原文摘要 · Abstract (English)
The recent advancements in Large Language Models(LLMs) have had a significant impact on a wide range of fields, from general domains to specialized areas. However, these advancements have also significantly increased the potential for malicious users to exploit harmful and jailbreak prompts for malicious attacks. Although there have been many efforts to prevent harmful prompts and jailbreak prompts, protecting LLMs from such malicious attacks remains an important and challenging task. In this paper, we propose QGuard, a simple yet effective safety guard method, that utilizes question prompting to block harmful prompts in a zero-shot manner. Our method can defend LLMs not only from text-based harmful prompts but also from multi-modal harmful prompt attacks. Moreover, by diversifying and modifying guard questions, our approach remains robust against the latest harmful prompts without fine-tuning. Experimental results show that our model performs competitively on both text-only and multi-modal harmful datasets. Additionally, by providing an analysis of question prompting, we enable a white-box analysis of user inputs. We believe our method provides valuable insights for real-world LLM services in mitigating security risks associated with harmful prompts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。