伪装成学术语言的恶意提示可诱使大模型输出偏见内容。
LLMs are Vulnerable to Malicious Prompts Disguised as Scientific Language
- 用伪科学表述诱导模型生成偏见言论
- 多款主流模型在学术伪装提示下偏见显著上升
- 适合关注AI安全与伦理的开发者和研究者
随着大语言模型（LLMs）在现实场景中的广泛应用，其可能传播有害内容的担忧日益增加。本文揭示，许多最先进的大模型对伪装成科学语言的恶意请求存在严重漏洞。实验表明，在GPT4o、GPT4o-mini、GPT-4、LLama3-405B-Instruct、Llama3-70B-Instruct、Cohere和Gemini等模型上，当提示故意将社会科学和心理学研究曲解为支持刻板印象有利性的证据时，模型的偏见与毒性显著升高。更令人担忧的是，这些模型能被诱导生成伪造的科学论据，声称偏见具有益处，可被恶意使用者用于系统性越狱。分析发现，提及作者与会议名称会增强提示说服力，且对话进行中偏见分数持续上升。研究呼吁对用于训练大模型的科学数据使用需更加审慎。
原文摘要 · Abstract (English)
As large language models (LLMs) have been deployed in various real-world settings, concerns about the harm they may propagate have grown. Various jailbreaking techniques have been developed to expose the vulnerabilities of these models and improve their safety. This work reveals that many state-of-the-art LLMs are vulnerable to malicious requests hidden behind scientific language. Specifically, our experiments with GPT4o, GPT4o-mini, GPT-4, LLama3-405B-Instruct, Llama3-70B-Instruct, Cohere, Gemini models demonstrate that, the models' biases and toxicity substantially increase when prompted with requests that deliberately misinterpret social science and psychological studies as evidence supporting the benefits of stereotypical biases. Alarmingly, these models can also be manipulated to generate fabricated scientific arguments claiming that biases are beneficial, which can be used by ill-intended actors to systematically jailbreak these strong LLMs. Our analysis studies various factors that contribute to the models' vulnerabilities to malicious requests in academic language. Mentioning author names and venues enhances the persuasiveness of models, and the bias scores increase as dialogues progress. Our findings call for a more careful investigation on the use of scientific data for training LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。