批量提问可诱导大模型出错，甚至触发隐藏后门。
An Investigation on Group Query Hallucination Attacks
- 将多个问题一次性输入模型，模拟真实对话中的上下文累积。
- 微调过的模型性能下降明显，预训练模型推理任务也受影响。
- 揭示潜在后门风险，适合安全研究者关注。
随着大语言模型（LLMs）的广泛应用，理解其在用户交互中的潜在失效模式至关重要。实际使用中，用户常在单次对话中提出多个问题。为此，本文提出群组查询攻击（Group Query Attack），通过同时呈现一组查询来模拟这一场景，探究连续提示带来的上下文累积对模型输出的影响。实验发现，该攻击显著降低特定任务微调模型的性能；同时证明其可能触发模型潜在后门。此外，该攻击对预训练和对齐模型在数学推理、代码生成等需要推理的任务中同样有效。
原文摘要 · Abstract (English)
With the widespread use of large language models (LLMs), understanding their potential failure modes during user interactions is essential. In practice, users often pose multiple questions in a single conversation with LLMs. Therefore, in this study, we propose Group Query Attack, a technique that simulates this scenario by presenting groups of queries to LLMs simultaneously. We investigate how the accumulated context from consecutive prompts influences the outputs of LLMs. Specifically, we observe that Group Query Attack significantly degrades the performance of models fine-tuned on specific tasks. Moreover, we demonstrate that Group Query Attack induces a risk of triggering potential backdoors of LLMs. Besides, Group Query Attack is also effective in tasks involving reasoning, such as mathematical reasoning and code generation for pre-trained and aligned models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。