让AI安全代理通过知识库实时学习，提升漏洞挖掘与攻击策略的准确性。
CRAKEN: Cybersecurity LLM Agent with Knowledge-Based Execution
- 基于知识库动态检索与反思，实现任务分解与策略迭代。
- 在纽约大学CTF基准上准确率达22%，比之前方法高3个百分点。
- 适合研究自动化攻防系统与可扩展安全智能体的开发者。
大型语言模型（LLM）代理能够自动化网络安全任务并适应不断演变的安全环境，而无需重新设计。尽管已有模型在夺旗竞赛（CTF）中展现出网络安全能力，但仍存在两大局限：无法获取训练数据之外的最新安全知识，且难以将新知识融入复杂任务规划。基于知识的方法可通过技术理解增强自动化求解能力，以克服这些挑战。我们提出CRAKEN——一种基于知识的LLM代理框架，通过三个核心机制提升网络安全能力：任务关键信息的上下文分解、迭代式自我反思的知识检索，以及将知识提示注入以生成自适应攻击策略。多配置综合评估表明，相较于先前方法，CRAKEN在多阶段漏洞检测与利用方面表现更优。其可扩展架构为嵌入新安全知识提供了新范式。使用包含CTF解题报告的知识库，CRAKEN在NYU CTF Bench上达到22%的准确率，领先前人3个百分点，达成当前最优水平。在MITRE ATT&CK技术评估中，CRAKEN解决的技术占比比之前工作高出25%-30%，验证了基于知识执行的有效性。代码已开源：https://github.com/NYU-LLM-CTF/nyuctf_agents_craken。
原文摘要 · Abstract (English)
Large Language Model (LLM) agents can automate cybersecurity tasks and can adapt to the evolving cybersecurity landscape without re-engineering. While LLM agents have demonstrated cybersecurity capabilities on Capture-The-Flag (CTF) competitions, they have two key limitations: accessing latest cybersecurity expertise beyond training data, and integrating new knowledge into complex task planning. Knowledge-based approaches that incorporate technical understanding into the task-solving automation can tackle these limitations. We present CRAKEN, a knowledge-based LLM agent framework that improves cybersecurity capability through three core mechanisms: contextual decomposition of task-critical information, iterative self-reflected knowledge retrieval, and knowledge-hint injection that transforms insights into adaptive attack strategies. Comprehensive evaluations with different configurations show CRAKEN's effectiveness in multi-stage vulnerability detection and exploitation compared to previous approaches. Our extensible architecture establishes new methodologies for embedding new security knowledge into LLM-driven cybersecurity agentic systems. With a knowledge database of CTF writeups, CRAKEN obtained an accuracy of 22% on NYU CTF Bench, outperforming prior works by 3% and achieving state-of-the-art results. On evaluation of MITRE ATT&CK techniques, CRAKEN solves 25-30% more techniques than prior work, demonstrating improved cybersecurity capabilities via knowledge-based execution. We make our framework open source to public https://github.com/NYU-LLM-CTF/nyuctf_agents_craken.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。