首个专为卢旺达语设计的句子嵌入模型，显著提升语义相似度计算效果。
KinyaEmbed: Contrastive Sentence Embeddings for Kinyarwanda via Multi-Stage Curriculum Training
- 分四阶段课程训练，结合自动生成与人工筛选的双语数据增强表示能力
- 在卢旺达语语义相似度任务上达到0.7298的斯皮尔曼相关系数，超越现有模型41%
- 开源全部模型、标注数据与新基准，助力非洲语言研究
我们提出KinyaEmbed，首个专为卢旺达语（一种有丰富形态变化的班图语，使用者超1200万）设计的句子嵌入模型。现有多语言嵌入模型如LaBSE、mE5-large和OpenAI text-embedding-3-large在卢旺达语上表现不佳，因其预训练语料严重缺乏该语言数据。KinyaEmbed基于KinyaBERT-large，采用四阶段课程训练策略，使用MultipleNegativesRankingLoss（MNRL）：第一阶段利用约1.8万个来自卢旺达官方公报的释义对，结合三种温度尺度；第二阶段在715个由NLLB翻译的MNLI三元组上微调，学习蕴含结构；第三阶段通过英-卢旺达语OPUS-100翻译对对齐表示；第四阶段用质量阈值0.8过滤后的2,936个高质量对进一步精炼。评估在SemRel2024-rw上进行，并引入Wiki-RW-STS——一个从卢旺达维基百科构建的300对无污染的卢旺达语语义相似度新基准。七检查点集成（all5+23A*2，最终阶段权重加倍）在SemRel2024-rw上取得0.7298的斯皮尔曼相关系数，较mE5-large提升20.9%，较OpenAI text-embedding-3-large提升41.0%。KinyaEmbed还在文档聚类任务中获得最佳轮廓系数（0.2146）。所有检查点、筛选后的KinyaCOMET数据对及Wiki-RW-STS基准均公开可用。
原文摘要 · Abstract (English)
We present KinyaEmbed, the first dedicated sentence embedding model for Kinyarwanda, a morphologically rich Bantu language spoken by over 12 million people in Rwanda. Existing multilingual embedding models such as LaBSE, mE5-large, and OpenAI text-embedding-3-large perform poorly on Kinyarwanda due to severe under-representation in their pre-training corpora. KinyaEmbed is built on KinyaBERT-large and trained via a four-stage curriculum using MultipleNegativesRankingLoss (MNRL): Stage 1 leverages ~18,000 paraphrase pairs from the Official Gazette of Rwanda with three temperature scales; Stage 2 fine-tunes on 715 NLLB-translated MNLI triplets for entailment structure; Stage 3 aligns representations using English-Kinyarwanda OPUS-100 translation pairs; Stage 4 refines with 2,936 high-quality pairs filtered from KinyaCOMET at quality threshold 0.8. We evaluate on SemRel2024-rw and introduce Wiki-RW-STS, a new contamination-free Kinyarwanda STS benchmark of 300 pairs derived from Kinyarwanda Wikipedia. A seven-checkpoint ensemble (all5+23A*2, with the final stage double-weighted) achieves Spearman \r{ho}=0.7298 on SemRel2024-rw, surpassing mE5-large by 20.9% and OpenAI text-embedding-3-large by 41.0%. KinyaEmbed also achieves the best document clustering silhouette score (0.2146) across all evaluated models. All checkpoints, the KinyaCOMET filtered pairs, and the Wiki-RW-STS benchmark are publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。