用针对性数据训练大模型，大幅减少翻译幻觉。
Mitigating Hallucinated Translations in Large Language Models with Hallucination-focused Preference Optimization
- 构建专注幻觉的偏好数据集，让模型训练时学会避免幻觉。
- 在5个语种对上幻觉率平均降低96%，保持翻译质量。
- 零样本下对3个新语言幻觉减少89%，适合生产部署。
机器翻译正经历范式转变，微调的大语言模型（LLM）系统在性能上日益接近专为翻译设计的传统编码器-解码器模型。然而，基于LLM的系统生成幻觉的可能性更高，严重损害用户信任与安全。以往研究多聚焦于传统翻译模型的幻觉缓解，通常采用事后检测与重译的方法，但会增加部署复杂性和延迟。为此，本文提出在模型训练阶段内生性地缓解幻觉。具体而言，设计了一种生成幻觉导向偏好数据的框架。在这些偏好数据上微调LLM，使5个语言对上的幻觉率平均降低96%，同时保持整体翻译质量。在零样本设置下，对3个未见目标语言的平均幻觉率也降低了89%。
原文摘要 · Abstract (English)
Machine Translation (MT) is undergoing a paradigm shift, with systems based on fine-tuned large language models (LLM) becoming increasingly competitive with traditional encoder-decoder models trained specifically for translation tasks. However, LLM-based systems are at a higher risk of generating hallucinations, which can severely undermine user's trust and safety. Most prior research on hallucination mitigation focuses on traditional MT models, with solutions that involve post-hoc mitigation - detecting hallucinated translations and re-translating them. While effective, this approach introduces additional complexity in deploying extra tools in production and also increases latency. To address these limitations, we propose a method that intrinsically learns to mitigate hallucinations during the model training phase. Specifically, we introduce a data creation framework to generate hallucination focused preference datasets. Fine-tuning LLMs on these preference datasets reduces the hallucination rate by an average of 96% across five language pairs, while preserving overall translation quality. In a zero-shot setting our approach reduces hallucinations by 89% on an average across three unseen target languages.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。