通过剪枝语言层和方向，让翻译模型更轻更快。
CULL-MT: Compression Using Language and Layer pruning for Machine Translation
- 按重要性逐层剪枝，保留关键结构。
- 剪掉25%层后，翻译质量仅降0.9 spBLEU。
- 适合需要精简多语言模型的部署场景。
多语言机器翻译模型通过知识迁移提升性能，支持数百种语言并达到领先效果。但模型规模增大导致推理成本上升。在多数实际应用中，只需少数语言对。本文提出CULL-MT，基于结构化层剪枝与选定语言方向压缩模型。通过贪心策略识别并移除不重要层，并利用知识蒸馏与参数高效微调缓解性能损失。在NLLB-3.3B和LLaMA3.1-8B-Instruct上测试，在波斯语、法语、德语到英语的多向翻译任务中，NLLB-3.3B可剪枝25%层，仅下降0.9 spBLEU；而LLaMA3.1-8B-Instruct剪5层后下降2.0 spBLEU。
原文摘要 · Abstract (English)
Multilingual machine translation models often outperform traditional bilingual models by leveraging translation knowledge transfer. Recent advancements have led to these models supporting hundreds of languages and achieving state-of-the-art results across various translation directions. However, as these models grow larger, their inference operations become increasingly costly. In many use cases, there is no need to support such a wide range of language pairs, as translation is typically needed in only a few selected directions. In this paper, we present CULL-MT, a compression method for machine translation models based on structural layer pruning and selected language directions. Our approach identifies and prunes unimportant layers using a greedy strategy, then mitigates the impact by applying knowledge distillation from the original model along with parameter-efficient fine-tuning. We apply CULL-MT to the NLLB-3.3B and LLaMA3.1-8B-Instruct models. In a multi-way translation scenario (Persian, French, and German to English), we find the NLLB-3.3B model to be robust, allowing 25% of layers to be pruned with only a 0.9 spBLEU drop. However, LLaMA3.1-8B-Instruct is more sensitive, with a 2.0 spBLEU drop after pruning 5 layers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。