用孟加拉语转写数据提升稀缺语言查克马语的模型性能
Exploring Cross-Lingual Knowledge Transfer via Transliteration-Based MLM Fine-Tuning for Critically Low-resource Chakma Language
- 构建孟加拉语转写的查克马语语料库，用于掩码语言建模微调
- 多语言模型在转写数据上微调后达到73.54%词符准确率，困惑度低至2.90
- 适合关注小语种、跨语言迁移与低资源语言研究者
作为印欧语系语言中数据极少的查克马语，长期未被主流语言模型覆盖。本文从查克马文学中构建了一个上下文连贯的孟加拉语转写查克马语语料库，并经母语者验证。基于该数据集，我们对六种编码器型Transformer模型（包括mBERT、XLM-RoBERTa、DistilBERT、BanglaBERT、IndicBERT及DeBERTaV3）进行掩码语言建模微调。实验表明，微调后的多语言模型在孟加拉语转写查克马语上的表现优于原始预训练模型，最高达到73.54%的词符准确率，困惑度低至2.90。分析显示数据质量对模型性能影响显著，并揭示了光学字符识别系统在形态丰富的印地文字母中的局限性。研究证明，使用孟加拉语转写可有效支持查克马语的迁移学习，我们已公开该语料库，以推动低资源语言的多语言建模范式研究。
原文摘要 · Abstract (English)
As an Indo-Aryan language with limited available data, Chakma remains largely underrepresented in language models. In this work, we introduce a novel corpus of contextually coherent Bangla-transliterated Chakma, curated from Chakma literature, and validated by native speakers. Using this dataset, we fine-tune six encoder-based transformer models, including multilingual (mBERT, XLM-RoBERTa, DistilBERT), regional (BanglaBERT, IndicBERT), and monolingual English (DeBERTaV3) variants on masked language modeling (MLM) tasks. Our experiments show that fine-tuned multilingual models outperform their pre-trained counterparts when adapted to Bangla-transliterated Chakma, achieving up to 73.54% token accuracy and a perplexity as low as 2.90. Our analysis further highlights the impact of data quality on model performance and shows the limitations of OCR pipelines for morphologically rich Indic scripts. Our research demonstrates that Bangla-transliterated Chakma can be very effective for transfer learning for Chakma language, and we release our dataset to encourage further research on multilingual language modeling for low-resource languages.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。