系统梳理机器翻译中的知识蒸馏方法与应用,揭示其提升效率与质量的关键机制。
KD4MT: A Survey of Knowledge Distillation for Machine Translation
- 基于105篇论文归纳知识蒸馏在机器翻译中的方法分类与实践场景
- 发现蒸馏能提升翻译质量与推理效率,但可能加剧幻觉与偏见
- 提供选型指南和风险提示,适合关注模型压缩与生成质量的研究者
知识蒸馏(KD)作为自然语言处理中大型模型压缩的重要手段近年来备受关注。在机器翻译(MT)领域,KD不仅用于模型轻量化,更是一种通用的知识迁移机制,影响监督信号、翻译质量与运行效率。本综述系统分析了截至2025年10月1日的105篇相关论文,首先为非专家介绍机器翻译与知识蒸馏的基础概念,接着概述适用于MT的标准蒸馏方法。随后,从方法论贡献与实际应用两个维度对研究进展进行分类。定性与定量分析揭示了领域内共性趋势,指出评估标准不统一、关键研究空白等问题。同时提供具体场景下的蒸馏方法选择建议,并警示使用中可能引发的幻觉增强与偏见放大风险。最后探讨大模型(LLMs)对知识蒸馏在机器翻译领域的影响。为支持后续研究,我们公开维护一个数据库,汇总所调研方法的核心特征,并提供关键术语词汇表。
原文摘要 · Abstract (English)
Knowledge Distillation (KD) as a research area has gained a lot of traction in recent years as a compression tool to address challenges related to ever-larger models in NLP. Remarkably, Machine Translation (MT) offers a much more nuanced take on this narrative: in MT, KD also functions as a general-purpose knowledge transfer mechanism that shapes supervision and translation quality as well as efficiency. This survey synthesizes KD for MT (KD4MT) across 105 papers (through October 1, 2025). We begin by introducing both MT and KD for non-experts, followed by an overview of the standard KD approaches relevant to MT applications. Subsequently, we categorize advances in the KD4MT literature based on (i) their methodological contributions and (ii) their practical applications. Our qualitative and quantitative analyses identify common trends in the field and highlight key research gaps as well as the absence of unified evaluation practice for KD methods in MT. We further provide practical guidelines for selecting a KD method in concrete settings and highlight potential risks associated with the application of KD to MT such as increased hallucination and bias amplification. Finally, we discuss the role of LLMs in re-shaping the KD4MT field. To support further research, we complement our survey with a publicly available database summarizing the main characteristics of the surveyed KD methods and a glossary of key terms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。