用剪枝提升mBERT效率，实现低资源果阿语隐喻与习语分类
Pruning for Performance: Efficient Idiom and Metaphor Classification in Low-Resource Konkani Using mBERT
- 融合mBERT、LSTM与线性分类器，结合梯度剪枝优化模型
- 隐喻分类准确率达78%，习语分类达83%
- 适合低资源语言的轻量化NLP工具开发
本文针对自然语言处理系统在低资源语言（如果阿语）中处理比喻语言的挑战，提出一种混合模型，结合预训练多语言BERT（mBERT）、双向LSTM和线性分类器，并在本研究构建的新标注数据集上进行微调。为提升模型效率，采用基于梯度的注意力头剪枝策略。剪枝后模型在隐喻分类任务上达到78%准确率，应用于现有习语分类任务时获得83%准确率。结果表明，注意力头剪枝在构建低资源语言高效NLP工具方面具有显著效果。
原文摘要 · Abstract (English)
In this paper, we address the persistent challenges that figurative language expressions pose for natural language processing (NLP) systems, particularly in low-resource languages such as Konkani. We present a hybrid model that integrates a pre-trained Multilingual BERT (mBERT) with a bidirectional LSTM and a linear classifier. This architecture is fine-tuned on a newly introduced annotated dataset for metaphor classification, developed as part of this work. To improve the model's efficiency, we implement a gradient-based attention head pruning strategy. For metaphor classification, the pruned model achieves an accuracy of 78%. We also applied our pruning approach to expand on an existing idiom classification task, achieving 83% accuracy. These results demonstrate the effectiveness of attention head pruning for building efficient NLP tools in underrepresented languages.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。