用微调的BanglaBERT检测孟加拉语煽动性暴力文本,准确率提升至0.63。
How Effectively Can BERT Models Interpret Context and Detect Bengali Communal Violent Text?
- 微调BanglaBERT并构建集成模型,应对数据不平衡问题。
- 集成模型在孟加拉语数据上达宏平均F1为0.63,优于单模型。
- 结合LIME与词向量相似度分析,揭示模型上下文理解局限。
网络仇恨言论的蔓延加剧了宗教、民族和社会群体间的冲突,威胁社会和谐。尽管至关重要,现有研究对煽动性暴力文本的分类仍关注不足。本研究聚焦社交平台上的孟加拉语文本,旨在提升此类文本的检测精度。我们提出一种针对该任务微调的BanglaBERT模型,宏平均F1得分为0.60。为缓解数据不平衡,数据集扩充了1,794个样本,支持训练和评估一个集成模型,其宏平均F1提升至0.63,验证了有效性。定量结果之外,定性分析发现模型在理解上下文时存在困难,即使高置信度预测也常出错。通过分析词向量余弦相似度,发现预训练模型难以区分密切相关的社群与非社群术语。应用LIME进一步揭示模型决策中的上下文误解区域,帮助解释分类错误。研究证明自然语言处理与可解释性工具在减少线上社群暴力方面的潜力,为未来提升准确性和社会影响提供了基础。
原文摘要 · Abstract (English)
The spread of cyber hatred has led to communal violence, fueling aggression and conflicts between various religious, ethnic, and social groups, posing a significant threat to social harmony. Despite its critical importance, the classification of communal violent text remains an underexplored area in existing research. This study aims to enhance the accuracy of detecting text that incites communal violence, focusing specifically on Bengali textual data sourced from social media platforms. We introduce a fine-tuned BanglaBERT model tailored for this task, achieving a macro F1 score of 0.60. To address the issue of data imbalance, our dataset was expanded by adding 1,794 instances, which facilitated the development and evaluation of a fine-tuned ensemble model. This ensemble model demonstrated an improved performance, achieving a macro F1 score of 0.63, thus highlighting its effectiveness in this domain. In addition to quantitative performance metrics, qualitative analysis revealed instances where the models struggled with context understanding, leading to occasional misclassifications, even when predictions were made with high confidence. Through analyzing the cosine similarity between words, we identified certain limitations in the pre-trained BanglaBERT models, particularly in their ability to distinguish between closely related communal and non-communal terms. To further interpret the model's decisions, we applied LIME, which helped to uncover specific areas where the model struggled in understanding context, contributing to errors in classification. These findings highlight the promise of NLP and interpretability tools in reducing online communal violence. Our work contributes to the growing body of research in communal violence detection and offers a foundation for future studies aiming to refine these techniques for better accuracy and societal impact.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。