通过修剪词汇表大幅降低多语言翻译模型内存占用，性能不降反升。
Efficient Multilingual Neural Machine Translation via Corpus-Driven Vocabulary Pruning: An English-Arabic Case Study
- 基于语料驱动的词汇剪枝+针对性微调，重构模型词汇结构。
- 词汇量从12.8万缩至1万，内存减少60%且性能无损。
- 适合资源受限场景下的高效多语言翻译部署。
将大型预训练多语言模型用于神经机器翻译面临主要挑战：过大的词汇表和嵌入层导致内存与计算开销过大。尽管现有压缩方法如剪枝、量化和知识蒸馏可减少参数冗余，但大多保留原始词汇结构，未能解决核心效率瓶颈。本文提出一种通用优化框架，结合词汇剪枝与针对性微调协议，应用于M2M100、NLLB-200、mBART-50三种模型，在英阿语言对上评估。该方法将词汇量从超过128,000缩减至约10,000词，实现60%内存节省，且性能无下降。结果表明，优化后的多语言模型可达到或超越专用双语基线表现。特别是剪枝并微调的M2M100模型，获得42.04的BLEU分数（对比OPUS-MTen-ar双语模型的44.59），在COMET指标上显著优于后者（0.8730 vs 0.7911），体现更强的语义准确性和流畅性。
原文摘要 · Abstract (English)
The adoption of large pre-trained multilingual models for neural machine translation (MNMT) faces a major challenge: excessive memory and computational consumption due to overly large vocabularies and embedding layers. Although existing compression methods like pruning, quantization and knowledge distillation reduce parameter redundancy, they mainly preserve the structure of the original vocabulary, thereby leaving a major source of inefficiency unresolved. We propose in this paper a general optimization framework that combines a vocabulary pruning method with a targeted fine-tuning protocol for MNMT models. We evaluate the proposed framework using three models (M2M100, NLLB-200, mBART-50) on the English-Arabic language pair. Our approach reduces the vocabulary size from over 128,000 to approximately 10,000 tokens, enabling a 60% memory saving without any loss in performance. Results show that optimized multilingual models can match or exceed the performance of dedicated bilingual baselines. In particular, the pruned and fine-tuned M2M100 model achieves a competitive BLEU score of 42.04 (against 44.59 for the OPUS-MTen- ar bilingual model) while it significantly outperforms it on the COMET metric (0.8730 vs 0.7911) revealing superior semantic adequacy and fluency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。