用合成数据和两阶段微调提升大模型文档级翻译质量
Enhancing Document-Level Machine Translation via Filtered Synthetic Corpora and Two-Stage LLM Adaptation
- 用大模型将摘要转为文档级双语数据并多指标过滤
- 先在句级数据上预训练，再在高质量文档数据上精调
- 适合关注长文本一致性与生成质量的研究者
在机器翻译中，大语言模型（LLMs）相比传统编码器-解码器系统表现欠佳，应用有限。然而，LLMs 在建模上下文信息方面优势明显，特别适合对句子间连贯性要求高的文档级翻译任务。尽管潜力巨大，当前基于 LLM 的文档级机器翻译仍面临两大挑战：(1) 缺乏大规模、高质量的文档级平行语料；(2) 模型生成时易出现幻觉和遗漏。为此，本文提出一种两阶段微调策略，利用 LLM 增强文档级数据。首先，通过 LLM 将摘要数据转换为文档级双语数据，并采用 sacreBLEU、COMET 及 LaBSE 余弦相似度等多指标进行过滤，提升数据质量；最后，采用两阶段微调：先在丰富的句级机器翻译资源上微调，再在过滤后的文档级语料上进一步优化。
原文摘要 · Abstract (English)
In Machine Translation, Large Language Models (LLMs) have generally underperformed compared to conventional encoder-decoder systems and thus see limited adoption. However, LLMs excel at modeling contextual information, making them a natural fit for document-level translation tasks where coherence across sentences is crucial. Despite this potential, document-level MT with LLMs faces two key challenges: (1) the scarcity of large-scale, high-quality document-level parallel data; and (2) the propensity of LLMs to introduce hallucinations and omissions during generation. To address these challenges, we propose a two-stage fine-tuning strategy leveraging LLM-augmented document-level data. First, we augment data by converting summarization data into document-level parallel data using a LLM, and then filter it using multiple metrics, leveraging sacreBLEU, COMET, and LaBSE-based cosine similarity-to improve data quality. Finally, we employ a two-stage fine-tuning strategy: first fine-tuning on the abundant sentence-level MT resources, and then on the filtered document-level corpus.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。