用少量参数微调模型，高效检测孟加拉语仇恨言论
Parameter-Efficient Fine-Tuning for Low-Resource Languages: A Comparative Study of LLMs for Bengali Hate Speech Detection
- 仅训练不足1%参数，实现低资源语言的高效微调
- Llama-3.2-3B模型在BD-SHS数据集上达92.23%最高F1分数
- 可在单张消费级显卡上运行，适合资源有限的研究者
孟加拉语社交媒体平台上的仇恨言论激增，尤其影响女性与青少年。尽管已有BD-SHS等标注数据集支持评估，但多数方法依赖计算成本高昂的全模型微调或专有API。本文首次将参数高效微调（PEFT）应用于孟加拉语仇恨言论检测，采用LoRA与QLoRA技术。在包含50,281条标注评论的BD-SHS数据集上，对三款指令微调的大语言模型——Gemma-3-4B、Llama-3.2-3B和Mistral-7B进行微调，每模型仅训练少于1%参数，可在单张消费级GPU上完成实验。结果表明，Llama-3.2-3B取得92.23%最高F1分数，其次为Mistral-7B的88.94%和Gemma-3-4B的80.25%。研究证实PEFT是孟加拉语及类似低资源语言的可行且可复现策略。
原文摘要 · Abstract (English)
Bengali social media platforms have witnessed a sharp increase in hate speech, disproportionately affecting women and adolescents. While datasets such as BD-SHS provide a basis for structured evaluation, most prior approaches rely on either computationally costly full-model fine-tuning or proprietary APIs. This paper presents the first application of Parameter-Efficient Fine-Tuning (PEFT) for Bengali hate speech detection using LoRA and QLoRA. Three instruction-tuned large language models - Gemma-3-4B, Llama-3.2-3B, and Mistral-7B - were fine-tuned on the BD-SHS dataset of 50,281 annotated comments. Each model was adapted by training fewer than 1% of its parameters, enabling experiments on a single consumer-grade GPU. The results show that Llama-3.2-3B achieved the highest F1-score of 92.23%, followed by Mistral-7B at 88.94% and Gemma-3-4B at 80.25%. These findings establish PEFT as a practical and replicable strategy for Bengali and related low-resource languages.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。