融合多种语言特征与机器学习,提升原住民语言翻译评估准确率
FUSE : A Ridge and Random Forest-Based Metric for Evaluating MT in Indigenous Languages
- 结合词汇、语音、语义等多维特征,用岭回归与梯度提升建模
- 在美洲原住民语言翻译评估中,皮尔逊相关性领先于现有方法
- 适合低资源、形态复杂的语言翻译质量评估场景
本文提出RaaVa团队在2025年AmericasNLP共享任务3中的获奖方案,针对美洲原住民语言的机器翻译自动评估。所提FUSE评分系统融合岭回归与梯度提升,整合词法、语音、语义及模糊词元相似性特征。该框架利用多语言句向量与音系编码,结合人工标注开发集训练监督模型,并在留出测试集上评估。结果表明,FUSE在与人类评价的皮尔逊相关性与斯皮尔曼相关性上均表现最优,有效应对多合成、复杂形态与非标准拼写的挑战,为低资源语言翻译评估提供可靠且语言学合理的解决方案。
原文摘要 · Abstract (English)
This paper presents the winning submission of the RaaVa team to the AmericasNLP 2025 Shared Task 3 on Automatic Evaluation Metrics for Machine Translation (MT) into Indigenous Languages of America, where our system ranked first overall based on average Pearson correlation with the human annotations. We introduce Feature-Union Scorer (FUSE) for Evaluation, FUSE integrates Ridge regression and Gradient Boosting to model translation quality. In addition to FUSE, we explore five alternative approaches leveraging different combinations of linguistic similarity features and learning paradigms. FUSE Score highlights the effectiveness of combining lexical, phonetic, semantic, and fuzzy token similarity with learning-based modeling to improve MT evaluation for morphologically rich and low-resource languages. MT into Indigenous languages poses unique challenges due to polysynthesis, complex morphology, and non-standardized orthography. Conventional automatic metrics such as BLEU, TER, and ChrF often fail to capture deeper aspects like semantic adequacy and fluency. Our proposed framework, formerly referred to as FUSE, incorporates multilingual sentence embeddings and phonological encodings to better align with human evaluation. We train supervised models on human-annotated development sets and evaluate held-out test data. Results show that FUSE consistently achieves higher Pearson and Spearman correlations with human judgments, offering a robust and linguistically informed solution for MT evaluation in low-resource settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。