通过干扰推理步骤，让大模型在无感知下得出错误结论。
Stepwise Reasoning Error Disruption Attack of LLMs
- 在推理前几步悄悄引入错误，误导后续推导。
- 在4个数据集、4种模型上均有效，零样本和少样本皆适用。
- 攻击隐蔽且不改指令，适合研究模型安全与鲁棒性。
大语言模型在复杂推理任务中表现突出，但其推理过程的安全性和鲁棒性仍待深入研究。现有攻击方法受限于特定场景或缺乏隐蔽性，难以推广。为此，我们提出步进式推理错误扰乱攻击（SEED），通过在前期推理步骤中微妙注入错误，引导模型产生后续错误推导及最终错误答案。与以往方法不同，SEED兼容零样本与少样本设置，保持自然推理流程，并在不修改指令的前提下实现隐蔽执行。在四个数据集、四种不同模型上的大量实验验证了其有效性，揭示了大语言模型在推理过程中的脆弱性。这些发现强调了在实际应用中提升推理鲁棒性的必要性。代码已公开：https://github.com/Applied-Machine-Learning-Lab/SEED-Attack。
原文摘要 · Abstract (English)
Large language models (LLMs) have made remarkable strides in complex reasoning tasks, but their safety and robustness in reasoning processes remain underexplored. Existing attacks on LLM reasoning are constrained by specific settings or lack of imperceptibility, limiting their feasibility and generalizability. To address these challenges, we propose the Stepwise rEasoning Error Disruption (SEED) attack, which subtly injects errors into prior reasoning steps to mislead the model into producing incorrect subsequent reasoning and final answers. Unlike previous methods, SEED is compatible with zero-shot and few-shot settings, maintains the natural reasoning flow, and ensures covert execution without modifying the instruction. Extensive experiments on four datasets across four different models demonstrate SEED's effectiveness, revealing the vulnerabilities of LLMs to disruptions in reasoning processes. These findings underscore the need for greater attention to the robustness of LLM reasoning to ensure safety in practical applications. Our code is available at: https://github.com/Applied-Machine-Learning-Lab/SEED-Attack.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。