用《红楼梦》研究文化负载翻译，发现大模型在跨文化表达上仍存短板。
On the Systematic Challenges of Culturally Loaded Machine Translation: Dream of the Red Chamber as the Cultural Lens
- 基于《红楼梦》构建中日双语文化语料库，覆盖500个文化片段。
- 顶尖大模型在文化表达翻译上表现不佳，人工评价差异显著。
- 现有评测指标无法可靠衡量文化翻译质量，需新评估体系。
文化负载翻译对机器翻译（MT）构成独特挑战，因意义深植于社会文化语境，超越表层语言形式。尽管大型语言模型（LLMs）使MT系统在多数场景下达到人类水平，但其处理文化负载表达的能力仍待深入探索。本研究系统考察了基于LLM的MT系统在文化负载翻译中的挑战。我们从具有文化代表性的《红楼梦》语料库构建了一个中日双语数据集，涵盖500个不同文化类别的段落。通过综合评估协议，揭示三大挑战：(1) 任务挑战，前沿LLMs在文化内容翻译上表现出明显性能差距；(2) 人工评估挑战，评估者背景导致翻译判断存在显著分歧；(3) 自动评估挑战，广泛使用的评测指标无法可靠评估此类任务的翻译质量。这些发现为计算科学与语言学中的文化导向翻译研究提供了重要启示。
原文摘要 · Abstract (English)
Culturally loaded translation poses unique challenges for machine translation (MT), as meanings are deeply embedded in socio-cultural contexts beyond surface linguistic forms. Although large language models (LLMs) have enabled MT systems to achieve human-like quality in many scenarios, their ability to handle culturally loaded expressions remains underexplored. In this study, we systematically investigate the challenges posed by culturally loaded translation in LLM-based MT systems. We construct a Chinese-Japanese bilingual dataset from the culturally representative corpus Dream of the Red Chamber, containing 500 segments across diverse cultural categories. Using a comprehensive evaluation protocol, we reveal three main challenges: (1) task challenges, where frontier LLMs exhibit notable performance gaps and struggle with culturally loaded content; (2) human evaluation challenges, where evaluator backgrounds lead to substantial disagreement in translation judgments; and (3) automatic evaluation challenges, where widely used metrics fail to reliably assess translation quality for this task. These findings may offer valuable insights for culture-oriented translation research in both computational science and linguistics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。