测试ChatGPT日英翻译表现，发现文档级优于句子级。
ChatGPT as a Translation Engine: A Case Study on Japanese-English
- 用简单和增强提示对比，发现文档级翻译效果更优
- ChatGPT-3.5准确率高，但ChatGPT-4更流畅
- 性能可媲美主流商用翻译系统
本研究评估ChatGPT在日英翻译中的表现，比较了简单与增强提示策略，并与主流商用翻译引擎进行对比。通过自动评分和基于MQM的人工评估发现，文档级翻译优于句子级翻译。实验未明确显示增强提示优于简单提示。自动评估显示ChatGPT-3.5表现更优，但存在准确率（ChatGPT-3.5）与流畅性（ChatGPT-4）的权衡。最终结果表明，ChatGPT的翻译性能可与两个广泛使用的翻译系统相媲美。
原文摘要 · Abstract (English)
This study investigates ChatGPT for Japanese-English translation, exploring simple and enhanced prompts and comparing against commercially available translation engines. Performing both automatic and MQM-based human evaluations, we found that document-level translation outperforms sentence-level translation for ChatGPT. On the other hand, we were not able to determine if enhanced prompts performed better than simple prompts in our experiments. We also discovered that ChatGPT-3.5 was preferred by automatic evaluation, but a tradeoff exists between accuracy (ChatGPT-3.5) and fluency (ChatGPT-4). Lastly, ChatGPT yields competitive results against two widely-known translation systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。