arXiv:2412.05862cs.CL2024-12被引 8

对比开源大模型与专业翻译模型在医学领域的表现,发现后者仍更优。

Domain-Specific Translation with Open-Source Large Language Models: Resource-Oriented Analysis

  • 用医学领域数据测试4种语言方向的翻译能力
  • 3.3B参数的NLLB模型在多数方向超越7-8B大模型
  • 适合关注医疗翻译、低资源语言任务的研究者

本研究比较了开源自回归解码器型大语言模型(LLM)与面向任务的机器翻译(MT)模型在医学领域的域特定翻译性能。实验覆盖英法、英葡、英斯瓦希里、斯瓦希里英四种语言方向,涵盖不同资源水平。尽管近期进展显著,LLM在专业翻译上仍与多语言编码器-解码器模型(如NLLB-200)存在明显质量差距。结果表明,在三个方向上,NLLB-200 3.3B优于所有7-8B参数范围的评测LLM。微调虽能提升Mistral和Llama等模型表现,但仍不及微调后的NLLB-200 3.3B。研究强调在中低资源环境下,专用MT模型对高质量领域翻译仍至关重要。此外,更大尺寸的LLM优于8B版本,提示可通过针对性数据选择与知识蒸馏,预训练领域专用中等规模模型以提升专业翻译的质量与效率。

原文摘要 · Abstract (English)

In this work, we compare the domain-specific translation performance of open-source autoregressive decoder-only large language models (LLMs) with task-oriented machine translation (MT) models. Our experiments focus on the medical domain and cover four language directions with varied resource availability: English-to-French, English-to-Portuguese, English-to-Swahili, and Swahili-to-English. Despite recent advancements, LLMs demonstrate a significant quality gap in specialized translation compared to multilingual encoder-decoder MT models such as NLLB-200. Our results indicate that NLLB-200 3.3B outperforms all evaluated LLMs in the 7-8B parameter range across three out of the four language directions. While fine-tuning improves the performance of LLMs such as Mistral and Llama, these models still underperform compared to fine-tuned NLLB-200 3.3B models. Our findings highlight the ongoing need for specialized MT models to achieve high-quality domain-specific translation, especially in medium-resource and low-resource settings. Moreover, the superior performance of larger LLMs over their 8B variants suggests potential value in pre-training domain-specific medium-sized language models, employing targeted data selection and knowledge distillation approaches to enhance both quality and efficiency in specialized translation tasks.

机器翻译医学翻译大模型评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。