用知识图谱和智能代理提升网络安全情报问答准确率
Beyond RAG for Cyber Threat Intelligence: A Systematic Evaluation of Graph-Based and Agentic Retrieval
- 构建图谱与代理结合的检索系统,支持多跳推理
- 混合方法在多跳问题上比传统检索提升35%准确率
- 适合需要复杂关系分析的网络安全分析师使用
网络安全情报(CTI)分析师需对大量叙事型安全报告回答复杂问题。检索增强生成(RAG)系统帮助语言模型访问外部知识,但传统的向量检索在处理涉及威胁行为者、恶意软件和漏洞之间关系的查询时表现不佳,因相关证据常分散在多个文本片段中。知识图谱通过显式表示实体与关系,支持结构化多跳推理。然而,图谱检索、智能体检索及混合方法等不同范式相继出现,其假设和失败模式各异。本文系统评估了四种RAG架构在CTI分析中的表现:标准向量检索、基于CTI知识图谱的图谱检索、可修复失败图谱查询的智能体变体,以及结合图谱与文本检索的混合方法。在涵盖3,300个问答对的真实场景下进行评估,包括事实查找、多跳关系查询、分析师合成问题和无法回答的情况。结果表明,图谱增强在结构化事实查询中提升性能;混合图谱-文本方法在多跳问题上比向量RAG最高提升35%答案质量,且比纯图谱系统更稳定可靠。
原文摘要 · Abstract (English)
Cyber threat intelligence (CTI) analysts must answer complex questions over large collections of narrative security reports. Retrieval-augmented generation (RAG) systems help language models access external knowledge, but traditional vector retrieval often struggles with queries that require reasoning over relationships between entities such as threat actors, malware, and vulnerabilities. This limitation arises because relevant evidence is often distributed across multiple text fragments and documents. Knowledge graphs address this challenge by enabling structured multi-hop reasoning through explicit representations of entities and relationships. However, multiple retrieval paradigms, including graph-based, agentic, and hybrid approaches, have emerged with different assumptions and failure modes. It remains unclear how these approaches compare in realistic CTI settings and when graph grounding improves performance. We present a systematic evaluation of four RAG architectures for CTI analysis: standard vector retrieval, graph-based retrieval over a CTI knowledge graph, an agentic variant that repairs failed graph queries, and a hybrid approach combining graph queries with text retrieval. We evaluate these systems on 3,300 CTI question-answer pairs spanning factual lookups, multi-hop relational queries, analyst-style synthesis questions, and unanswerable cases. Results show that graph grounding improves performance on structured factual queries. The hybrid graph-text approach improves answer quality by up to 35 percent on multi-hop questions compared to vector RAG, while maintaining more reliable performance than graph-only systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。