arXiv:2411.06772cs.AI2024-11被引 3

融合对抗训练与预训练模型,提升电信诈骗文本分类准确率。

A Text Classification Model Combining Adversarial Training with Pre-trained Language Model and neural networks: A Case Study on Telecom Fraud Incident Texts

  • 用预训练模型提取语言特征,并通过FGM扰动嵌入层增强鲁棒性。
  • 在真实数据上达到83.9%分类准确率,显著优于人工分类效率。
  • 已部署于公安部门,适用于其他相似场景的智能分类任务。

一线警务人员需将电信诈骗报案文本划分为14类以实施精准防控,但数据量大、内容多样、表达不一,现有依赖人工分类的方式效率低。本文提出一种结合对抗训练与预训练语言模型及神经网络的文本分类模型。该模型利用语言学驱动的预训练模型提取三类语言特征,并采用快速梯度法(FGM)对生成的嵌入层进行扰动。随后,双向长短期记忆网络(Bi-LSTM)和卷积神经网络(CNN)分别捕捉上下文句法与局部语义信息。模型在运营部门提供的部分电信诈骗案例数据上训练后,达到83.9%的分类准确率。该模型已在实际部门部署,大幅减少人力投入,提升打击电信诈骗的效率。此外,由于其通用性,未来可拓展至其他应用场景。

原文摘要 · Abstract (English)

Front-line police officers often categorize all police call reported cases of Telecom Fraud into 14 subcategories to facilitate targeted prevention measures, such as precise public education. However, the associated data is characterized by its large volume, diverse information content, and variations in expression. Currently, there is a lack of efficient and accurate intelligent models to replace manual classification, which, while precise, is relatively inefficient. To address these challenges, this paper proposes a text classification model that combines adversarial training with Pre-trained Language Model and neural networks. The Linguistically-motivated Pre-trained Language Model model extracts three types of language features and then utilizes the Fast Gradient Method algorithm to perturb the generated embedding layer. Subsequently, the Bi-directional Long Short-Term Memory and Convolutional Neural Networks networks extract contextual syntactic information and local semantic information, respectively. The model achieved an 83.9% classification accuracy when trained on a portion of telecom fraud case data provided by the operational department. The model established in this paper has been deployed in the operational department, freeing up a significant amount of manpower and improving the department's efficiency in combating Telecom Fraud crimes. Furthermore, considering the universality of the model established in this paper, other application scenarios await further exploration.

文本分类对抗训练预训练模型电信诈骗

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。