用状态空间模型提升孟加拉语假新闻检测效率,兼顾性能与资源消耗。
BanglaMamba: Exploring State Space Models for Bangla Fake News Detection

- 采用Mamba架构的BanglaMamba模型,实现高效长文本处理。
- 性能接近自训练BERT(F1 0.9029),推理速度提升2.2倍。
- 适合边缘设备部署,尤其适合算力受限场景。
虚假新闻检测因在线平台和社交媒体中信息误导的快速传播而成为自然语言处理的重要任务。尽管基于Transformer的孟加拉语预训练模型BanglaBERT在文本分类中表现优异,但其二次计算复杂度使其在资源受限环境下处理长文档时效率较低。本文探索基于Mamba的状态空间模型(SSMs)作为替代方案,提出BanglaMamba,并与预训练的BanglaBERT及从零训练的CustomBERT进行对比。实验表明,BanglaBERT取得最高宏平均F1分数(0.9260),而BanglaMamba(0.9029)性能接近自训练模型(0.9057),且推理吞吐量高出约2.2倍,峰值GPU内存使用降低49%。跨数据集评估显示,BanglaBERT在外部数据上泛化能力更强,凸显大规模预训练的重要性。结果表明,基于Mamba的SSMs可为孟加拉语假新闻检测提供兼具竞争力与计算效率的替代方案,尤其适用于资源受限环境。
原文摘要 · Abstract (English)
Fake news detection has become an important Natural Language Processing (NLP) task due to the rapid spread of misinformation through online news platforms and social media. While transformer-based models such as BanglaBERT achieve strong performance for Bangla text classification, their quadratic computational complexity makes them less suitable for long-document processing in resource-constrained environments. This paper investigates Mamba-based State Space Models (SSMs) as an efficient alternative for Bangla fake news detection. We propose BanglaMamba and compare it with pre-trained BanglaBERT and a similarly configured BERT model trained from scratch. Experimental results show that BanglaBERT achieves the highest Macro-F1 score (0.9260), while BanglaMamba (0.9029) achieves performance comparable to the from-scratch CustomBERT (0.9057) despite using a different architecture. Meanwhile, BanglaMamba achieves approximately $2.2\times$ higher inference throughput and 49% lower inference peak GPU memory usage than the BERT-based models. Cross-dataset evaluation shows that BanglaBERT generalizes better to an external dataset, highlighting the importance of large-scale pretraining. These findings demonstrate that Mamba-based SSMs can provide a competitive and computationally efficient alternative to Transformer-based architectures for Bangla fake news detection, particularly in resource-constrained settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。