为阿姆哈拉语打造专用稠密检索模型,性能显著超越通用模型。
Optimized Text Embedding Models and Benchmarks for Amharic Passage Retrieval
- 基于阿姆哈拉语BERT和RoBERTa构建专用嵌入模型。
- 最佳模型相对基线提升17.6% MRR@10,且体积缩小13倍以上。
- 适合低资源语言信息检索研究者使用。
基于Transformer的预训练语言模型在多语言与跨语言检索中已取得进展,但对阿姆哈拉语等低资源、形态丰富的语言,其效果仍不充分,主要受限于数据稀缺与分词不当。本文提出基于阿姆哈拉语BERT和RoBERTa的专用稠密检索模型。其中,RoBERTa-Base-Amharic-Embed(110M参数)相较最强多语言基线Arctic Embed 2.0(568M参数)在MRR@10上提升17.6%,Recall@10提升9.86%。更紧凑的RoBERTa-Medium-Amharic-Embed(42M参数)在保持竞争力的同时,体积小于基线13倍以上。此外,基于ColBERT的晚期交互模型在所有评测中达到最高MRR@10(0.843)。我们系统性地对比了稀疏与稠密检索基线,揭示了低资源场景下的关键挑战,并强调语言特化的重要性。为推动低资源信息检索研究,我们公开发布数据集、代码与训练模型,地址:https://github.com/kidist-amde/amharic-ir-benchmarks。
原文摘要 · Abstract (English)
Neural retrieval methods using transformer-based pre-trained language models have advanced multilingual and cross-lingual retrieval. However, their effectiveness for low-resource, morphologically rich languages such as Amharic remains underexplored due to data scarcity and suboptimal tokenization. We address this gap by introducing Amharic-specific dense retrieval models based on pre-trained Amharic BERT and RoBERTa backbones. Our proposed RoBERTa-Base-Amharic-Embed model (110M parameters) achieves a 17.6% relative improvement in MRR@10 and a 9.86% gain in Recall@10 over the strongest multilingual baseline, Arctic Embed 2.0 (568M parameters). More compact variants, such as RoBERTa-Medium-Amharic-Embed (42M), remain competitive while being over 13x smaller. Additionally, we train a ColBERT-based late interaction retrieval model that achieves the highest MRR@10 score (0.843) among all evaluated models. We benchmark our proposed models against both sparse and dense retrieval baselines to systematically assess retrieval effectiveness in Amharic. Our analysis highlights key challenges in low-resource settings and underscores the importance of language-specific adaptation. To foster future research in low-resource IR, we publicly release our dataset, codebase, and trained models at https://github.com/kidist-amde/amharic-ir-benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。