arXiv:2603.25056cs.CRcs.AI2026-03被引 2

系统提示配置直接影响大模型邮件安全检测能力,错误配置会引入可被利用的漏洞。

The System Prompt Is the Attack Surface: How LLM Agent Configuration Shapes Security and Creates Exploitable Vulnerabilities

  • 通过11个模型、10种提示策略测试,发现提示与模型交互是安全关键变量。
  • 最优提示使钓鱼邮件检出率达93.7%,误报率仅3.8%,但易受信号反转攻击。
  • 越具体的提示反而降低鲁棒性,适合关注模型安全性与部署实用性的研究者。

系统提示配置可使大模型邮件代理的钓鱼邮件检测率从不足1%跃升至97%。我们通过PhishNChips研究了11个模型在10种提示策略下的表现,发现提示-模型交互是首要安全变量:同一模型在不同配置下钓鱼绕过率差异高达97%。优化提示以捕捉高预测性信号可提升基准性能,最高达93.7%召回率(3.8%假阳性率),但形成脆弱攻击面。特别是域名匹配策略在合法邮件域一致时表现良好,但攻击者通过注册匹配域名即可使其性能骤降。响应追踪显示98%的成功绕过均基于反向信号推理——模型遵循指令,但前提已失效。一个反直觉结论是:更具体的提示可能削弱已有能力模型,因其将多信号综合判断转为依赖可被操纵的单一信号。我们提出可部署性度量Safetility(惩罚假阳性),并认为缩小对抗差距需引入外部真实信息工具支持。

原文摘要 · Abstract (English)

System prompt configuration can make the difference between near-total phishing blindness and near-perfect detection in LLM email agents. We present PhishNChips, a study of 11 models under 10 prompt strategies, showing that prompt-model interaction is a first-order security variable: a single model's phishing bypass rate ranges from under 1% to 97% depending on how it is configured, while the false-positive cost of the same prompt varies sharply across models. We then show that optimizing prompts around highly predictive signals can improve benchmark performance, reaching up to 93.7% recall at 3.8% false positive rate, but also creates a brittle attack surface. In particular, domain-matching strategies perform well when legitimate emails mostly have matched sender and URL domains, yet degrade sharply when attackers invert that signal by registering matching infrastructure. Response-trace analysis shows that 98% of successful bypasses reason in ways consistent with the inverted signal: the models are following the instruction, but the instruction's core assumption has become false. A counter-intuitive corollary follows: making prompts more specific can degrade already-capable models by replacing broader multi-signal reasoning with exploitable single-signal dependence. We characterize the resulting tension between detection, usability, and adversarial robustness as a navigable tradeoff, introduce Safetility, a deployability-aware metric that penalizes false positives, and argue that closing the adversarial gap likely requires tool augmentation with external ground truth.

LLM安全提示工程钓鱼检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。