用权重变化判断语言切换时机，提升少数据多语言翻译效果
Can the Variation of Model Weights be used as a Criterion for Self-Paced Multilingual NMT?
- 根据Transformer各层权重的平滑KL散度决定每批训练的语言
- 相比交替单语批次，翻译质量与收敛速度均更优
- 适合资源稀缺场景下的多语言模型训练
当训练数据稀缺时，多对一神经机器翻译系统优于一对一系统。本文设计并测试了一种新算法，用于在训练此类系统时选择小批量的语种。当模型权重变化不显著时（通过Transformer网络所有层的平滑KL散度衡量），该算法会改变小批量的语言。实验表明，该方法在翻译质量（以BLEU和COMET衡量）和收敛速度方面优于交替单语批次，但不及打乱批次。
原文摘要 · Abstract (English)
Many-to-one neural machine translation systems improve over one-to-one systems when training data is scarce. In this paper, we design and test a novel algorithm for selecting the language of minibatches when training such systems. The algorithm changes the language of the minibatch when the weights of the model do not evolve significantly, as measured by the smoothed KL divergence between all layers of the Transformer network. This algorithm outperforms the use of alternating monolingual batches, but not the use of shuffled batches, in terms of translation quality (measured with BLEU and COMET) and convergence speed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。