arXiv:2411.05442cs.IR2024-11被引 18

IntellBot用检索增强生成技术,让安全专家和公众秒懂最新威胁情报。

IntellBot: Retrieval Augmented LLM Chatbot for Cyber Threat Knowledge Delivery

  • 基于大模型与RAG架构,从多源数据构建安全知识库。
  • BERT得分超0.8,余弦相似度达0.8~1,回应准确率高。
  • 适合安全人员快速查漏洞、做响应,也助公众理解威胁。

在快速演进的网络安全领域,智能聊天机器人正日益重要。人工智能、机器学习与自然语言处理使这些机器人能应答用户咨询并提供威胁情报,让安全知识对专业人员和公众都触手可及。传统规则型聊天机器人灵活性不足,难以适应交互变化;而基于大语言模型的聊天机器人可在多领域提供上下文相关的信息,并适应动态对话。本文提出IntellBot,一个基于前沿技术(如大语言模型、Langchain和检索增强生成模型)的先进网络安全聊天机器人,通过整合多元数据源构建涵盖已知漏洞、近期攻击与新兴威胁的综合知识库,提供定制化响应,成为网络安全洞察的核心枢纽。该系统实现威胁情报的即时获取与资源推送,提升应急响应效率与整体安全水平,节省时间并赋能用户掌握最佳实践。我们采用两阶段评估策略分析协作效果:间接方法获得的BERT分数高于0.8,余弦相似度在0.8至1之间,验证了回复准确性;此外,使用RAGAS评估RAG模型,所有指标均超过0.77,表明系统有效性显著。

原文摘要 · Abstract (English)

In the rapidly evolving landscape of cyber security, intelligent chatbots are gaining prominence. Artificial Intelligence, Machine Learning, and Natural Language Processing empower these chatbots to handle user inquiries and deliver threat intelligence. This helps cyber security knowledge readily available to both professionals and the public. Traditional rule-based chatbots often lack flexibility and struggle to adapt to user interactions. In contrast, Large Language Model-based chatbots offer contextually relevant information across multiple domains and adapt to evolving conversational contexts. In this work, we develop IntellBot, an advanced cyber security Chatbot built on top of cutting-edge technologies like Large Language Models and Langchain alongside a Retrieval-Augmented Generation model to deliver superior capabilities. This chatbot gathers information from diverse data sources to create a comprehensive knowledge base covering known vulnerabilities, recent cyber attacks, and emerging threats. It delivers tailored responses, serving as a primary hub for cyber security insights. By providing instant access to relevant information and resources, this IntellBot enhances threat intelligence, incident response, and overall security posture, saving time and empowering users with knowledge of cyber security best practices. Moreover, we analyzed the performance of our copilot using a two-stage evaluation strategy. We achieved BERT score above 0.8 by indirect approach and a cosine similarity score ranging from 0.8 to 1, which affirms the accuracy of our copilot. Additionally, we utilized RAGAS to evaluate the RAG model, and all evaluation metrics consistently produced scores above 0.77, highlighting the efficacy of our system.

安全聊天机器人RAG威胁情报LLM

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。