用LSTM+CRF实现无需领域知识的网络安全实体识别。
LSTM Recurrent Neural Networks for Cybersecurity Named Entity Recognition
- 采用LSTM与CRF结合，自动学习文本特征
- 在合理规模语料上超越现有最佳方法
- 适合无专家背景的团队快速部署
如今，从博客、文章等非结构化网络来源自动及时地转换网络安全信息为更正式的表示形式，已成为该领域诸多应用的必要需求。命名实体识别（NER）是迈向这一目标的关键早期步骤，涉及在技术文档中检测产品、版本、攻击名称等关键领域实体。尽管在信息抽取领域通常被视为简单任务，但在网络安全领域因其实体结构复杂而极具挑战性。当前最先进的方法依赖于耗时且人力密集的特征工程，需描述实体属性、上下文、领域知识和语言特征。本文提出的模型具有领域无关性，不依赖网络安全领域特定特征，因此无需专家进行特征工程。该方法基于长短期记忆网络（LSTM）与条件随机场（CRF）相结合。实验结果表明，在具备一定规模的标注语料上，该方法优于现有最佳方法。
原文摘要 · Abstract (English)
The automated and timely conversion of cybersecurity information from unstructured online sources, such as blogs and articles to more formal representations has become a necessity for many applications in the domain nowadays. Named Entity Recognition (NER) is one of the early phases towards this goal. It involves the detection of the relevant domain entities, such as product, version, attack name, etc. in technical documents. Although generally considered a simple task in the information extraction field, it is quite challenging in some domains like cybersecurity because of the complex structure of its entities. The state of the art methods require time-consuming and labor intensive feature engineering that describes the properties of the entities, their context, domain knowledge, and linguistic characteristics. The model demonstrated in this paper is domain independent and does not rely on any features specific to the entities in the cybersecurity domain, hence does not require expert knowledge to perform feature engineering. The method used relies on a type of recurrent neural networks called Long Short-Term Memory (LSTM) and the Conditional Random Fields (CRFs) method. The results we obtained showed that this method outperforms the state of the art methods given an annotated corpus of a decent size.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。