通过重启式课程学习提升机器翻译偏好优化效果
CLewR: Curriculum Learning with Restarts for Machine Translation Preference Learning
- 采用多轮难易递进的课程学习策略，动态调整训练数据顺序
- 在Gemma2/Qwen2.5/Llama3.1上实现稳定性能提升，避免简单样本遗忘
- 适合关注翻译模型训练策略与偏好学习的开发者和研究者
大语言模型在零样本多语言机器翻译中表现出色。后续工作通过偏好优化进一步提升了翻译性能，但对训练过程中数据样本的输入顺序这一关键因素仍关注不足。本文将课程学习引入多种先进偏好优化算法，提出一种带重启的新型课程学习策略（CLewR），通过多次重复从易到难的训练流程，有效缓解了简单样本的灾难性遗忘问题。实验表明，该方法在Gemma2、Qwen2.5、Llama3.1等多个模型族及不同偏好优化技术下均取得一致提升。代码已公开于https://github.com/alexandra-dragomir/CLewR。
原文摘要 · Abstract (English)
Large language models (LLMs) have demonstrated competitive performance in zero-shot multilingual machine translation (MT). Some follow-up works further improved MT performance via preference optimization, but they leave a key aspect largely underexplored: the order in which data samples are given during training. We address this topic by integrating curriculum learning into various state-of-the-art preference optimization algorithms to boost MT performance. We introduce a novel curriculum learning strategy with restarts (CLewR), which reiterates easy-to-hard curriculum multiple times during training to effectively mitigate the catastrophic forgetting of easy examples. We demonstrate consistent gains across several model families (Gemma2, Qwen2.5, Llama3.1) and preference optimization techniques. We publicly release our code at https://github.com/alexandra-dragomir/CLewR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。