纯提示词设计让AI在黑客竞赛中达到95%表现，远超前人。
Hacking CTFs with Plain Agents
- 用提示词+工具调用+多次尝试，简单策略突破安全挑战
- 在InterCode-CTF上达95%准确率，超越此前最高72%
- 适合对安全测试或大模型应用感兴趣的开发者
我们采用简单的LLM代理设计，在高中级别渗透测试基准InterCode-CTF上实现了95%的性能表现。该方法仅依赖提示工程、工具使用和多次尝试，显著优于之前Phuong等人（2024）的29%和Abramovich等人（2024）的72%。结果表明，当前大模型在进攻性网络安全任务上已超越高中生水平。其真实攻击能力仍被低估：我们提出的ReAct&Plan提示策略可在1-2轮内解决多数问题，无需复杂工程或高级封装。
原文摘要 · Abstract (English)
We saturate a high-school-level hacking benchmark with plain LLM agent design. Concretely, we obtain 95% performance on InterCode-CTF, a popular offensive security benchmark, using prompting, tool use, and multiple attempts. This beats prior work by Phuong et al. 2024 (29%) and Abramovich et al. 2024 (72%). Our results suggest that current LLMs have surpassed the high school level in offensive cybersecurity. Their hacking capabilities remain underelicited: our ReAct&Plan prompting strategy solves many challenges in 1-2 turns without complex engineering or advanced harnessing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。