用语言模型+半监督集成法,跨平台识别暗网和社交平台的非法交易内容。
A Language Model-Driven Semi-Supervised Ensemble Framework for Illicit Market Detection Across Deep/Dark Web and Social Platforms
- 融合微调的ModernBERT与人工特征,捕捉隐蔽语言和异常模式。
- 在三个数据集上准确率达96.5%,F1-score达93.5%,优于多个基线模型。
- 适合网络安全、执法机构用于实时监控非法交易信息。
非法市场正越来越多地转向互联网的隐蔽区域,包括深网、暗网以及Telegram、Reddit、Pastebin等平台。这些渠道支持毒品、武器、被盗凭证等违禁品的匿名交易。由于标注数据有限、非法语言不断演变及在线来源结构差异大,内容检测与分类仍具挑战。本文提出一种分层分类框架,结合微调的语言模型与半监督集成学习策略,实现跨平台非法市场内容的检测与分类。我们采用ModernBERT(针对长文档的Transformer模型)在深网页面、Telegram频道、Subreddit帖子和Pastebin贴文等领域的数据上进行微调,以捕捉专业术语与模糊语言模式。同时引入人工设计特征,如文档结构、嵌入式模式(比特币地址、邮箱、IP)及元数据,补充语言模型嵌入。分类流程分两阶段:第一阶段使用基于熵加权投票的XGBoost、随机森林与SVM半监督集成,检测销售相关文档;第二阶段进一步将销售内容分类为毒品、武器或凭证。在三个数据集(包括多源语料库、DUTA和CoDA)上的实验表明,该模型优于BERT、ModernBERT、DarkBERT、ALBERT、Longformer和BigBird等基线模型,准确率0.96489,F1-score 0.93467,TMCC 0.95388,展现出强泛化能力、弱监督下的鲁棒性及真实场景中的有效性。
原文摘要 · Abstract (English)
Illegal marketplaces have increasingly shifted to concealed parts of the internet, including the deep and dark web, as well as platforms such as Telegram, Reddit, and Pastebin. These channels enable the anonymous trade of illicit goods including drugs, weapons, and stolen credentials. Detecting and categorizing such content remains challenging due to limited labeled data, the evolving nature of illicit language, and the structural heterogeneity of online sources. This paper presents a hierarchical classification framework that combines fine-tuned language models with a semi-supervised ensemble learning strategy to detect and classify illicit marketplace content across diverse platforms. We extract semantic representations using ModernBERT, a transformer model for long documents, finetuned on domain-specific data from deep and dark web pages, Telegram channels, Subreddits, and Pastebin pastes to capture specialized jargon and ambiguous linguistic patterns. In addition, we incorporate manually engineered features such as document structure, embedded patterns including Bitcoin addresses, emails, and IPs, and metadata, which complement language model embeddings. The classification pipeline operates in two stages. The first stage uses a semi-supervised ensemble of XGBoost, Random Forest, and SVM with entropy-based weighted voting to detect sales-related documents. The second stage further classifies these into drug, weapon, or credential sales. Experiments on three datasets, including our multi-source corpus, DUTA, and CoDA, show that our model outperforms several baselines, including BERT, ModernBERT, DarkBERT, ALBERT, Longformer, and BigBird. The model achieves an accuracy of 0.96489, an F1-score of 0.93467, and a TMCC of 0.95388, demonstrating strong generalization, robustness under limited supervision, and effectiveness in real-world illicit content detection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。