arXiv:2606.09767cs.CLcs.AI2026-06中稿 · the 29th Internati…

用合成数据+高效微调,让濒危语言实现机器翻译

Data Synthesis and Parameter-Efficient Fine-Tuning for Low-Resource NMT: A Case Study on Q'eqchi' Mayan

  • 将社区词典转为合成语料,用LoRA微调mT5模型
  • 合成数据使模型在领域内达到BLEU 42.02,结构掌握好
  • 需真实语料做后期优化,否则会僵化于模板

针对数字资源极度匮乏的原住民语言,传统依赖网络爬取平行语料常引发数据主权问题。本研究以奇基查·马雅语(Q'eqchi' Mayan)为例,提出一种无需爬取目标语言平行文本的数据合成方法:将社区提供的词典转化为大规模合成语料,基于mT5-base模型,采用参数高效微调(PEFT)技术中的LoRA适配器进行训练。领域内评估显示模型获得高结构习得能力(BLEU 42.02),有效掌握复杂的黏着形态与VOS语序。然而,与自然词表对比时,模型表现极差(BLEU 0.59),虽语法正确但缺乏词汇语义根基。模型对合成模板的结构变体过度拟合,尽管生成语义熵高,仍难以适应自然语言的句法灵活性,导致有机输入被强行塞入僵化模式。消融实验表明,多任务学习架构引发负迁移,因辅助任务争夺有限的LoRA参数容量,造成模型过度优化于合成标记而牺牲自然语感。结论是:合成数据可作为高效的结构预训练工具,但必须通过课程学习引入真实语料进行语义精修。

原文摘要 · Abstract (English)

Neural machine translation for digitally low-resource Indigenous languages is often hindered by extreme data scarcity, prompting reliance on extractive web-scraping. To ensure data sovereignty, this study introduces a data synthesis methodology to bootstrap NMT models without scraping target-language parallel text. Focusing on Q'eqchi' Mayan, we transformed community-sourced dictionaries into a massive synthetic corpus, utilizing Parameter-Efficient Fine-Tuning (PEFT) via LoRA adapters on an mT5-base model. In-domain evaluation demonstrates high structural acquisition (BLEU 42.02), proving that synthetic constraints effectively teach complex agglutinative morphology and VOS word order. However, evaluation against an organic glossary reveals a structural-semantic gap (BLEU 0.59), where the model maintains grammatical integrity but lacks the lexical grounding of natural language. The model exhibits overfitting to the constrained structural variance of the synthetic templates; despite high semantic entropy in the pipeline, it struggles with the syntactic fluidity of natural language, forcing organic inputs into rigid learned patterns. Furthermore, an ablation study utilizing a Multi-Task Learning architecture resulted in negative transfer, suggesting that auxiliary tasks competed for limited parameter capacity within the LoRA adapters, causing over-optimization for synthetic markers at the expense of organic flexibility. Ultimately, we establish that synthetic bootstrapping is a highly effective structural primer, but requires authentic data for semantic refinement via Curriculum Learning.

低资源翻译合成数据参数高效微调濒危语言

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。