代码智能体可自动挖出大模型隐藏指令,暴露安全漏洞。
Just Ask: Curious Code Agents Reveal System Prompts in Frontier LLMs
- 用自主交互探索策略,无需人工提示或标注数据。
- 在41个商用模型上成功恢复完整系统提示,成功率高。
- 适合关注AI安全、模型防御的研究者与工程师。
基于大语言模型的自主代码代理正通过工具使用、长程推理和自我驱动交互重塑软件与AI开发。然而,这种自主性引入了前所未有的安全风险:代理交互从根本上扩大了大模型的攻击面,使得系统提示的系统性探测与恢复成为可能。我们发现系统提示提取是代码代理固有的新兴漏洞,并提出 extbf{ extsc{JustAsk}}——一个自演化框架,仅通过交互即可自主发现有效的提取策略。不同于以往依赖手工提示或数据集的攻击方法, extsc{JustAsk} 无需人工构造提示、标签监督或特权访问,仅需标准用户交互。它将提取问题建模为在线探索任务,采用基于置信上界(UCB)的策略选择机制,并构建涵盖原子探测与高层编排的分层技能空间。这些技能利用系统指令泛化不完善以及助人与安全之间的内在张力。在跨多家供应商的41个黑盒商用模型上评估显示, extsc{JustAsk} 均能持续实现完整或近乎完整的系统提示恢复,揭示出设计与架构层面的共性漏洞。结果表明,系统提示已成为现代代理系统中关键却缺乏保护的攻击面。
原文摘要 · Abstract (English)
Autonomous code agents built on large language models are reshaping software and AI development through tool use, long-horizon reasoning, and self-directed interaction. However, this autonomy introduces a previously unrecognized security risk: agentic interaction fundamentally expands the LLM attack surface, enabling systematic probing and recovery of hidden system prompts that guide model behavior. We identify system prompt extraction as an emergent vulnerability intrinsic to code agents and present \textbf{\textsc{JustAsk}}, a self-evolving framework that autonomously discovers effective extraction strategies through interaction alone. Unlike prior prompt-engineering or dataset-based attacks, \textsc{JustAsk} requires no handcrafted prompts, labeled supervision, or privileged access beyond standard user interaction. It formulates extraction as an online exploration problem, using Upper Confidence Bound-based strategy selection and a hierarchical skill space spanning atomic probes and high-level orchestration. These skills exploit imperfect system-instruction generalization and inherent tensions between helpfulness and safety. Evaluated on \textbf{41} black-box commercial models across multiple providers, \textsc{JustAsk} consistently achieves full or near-complete system prompt recovery, revealing recurring design- and architecture-level vulnerabilities. Our results expose system prompts as a critical yet largely unprotected attack surface in modern agent systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。