arXiv:2601.22931cs.CL2026-01被引 3

构建中文社交文本翻译基准,测试模型对网络用语和风格的把握能力。

Benchmarking Machine Translation on Chinese Social Media Texts

  • 设计两个专家标注子集:趣味帖子含大量新词,社交短句重情感与风格。
  • 发现20多个主流模型在俚语翻译成功率上差异显著,风格保留率普遍偏低。
  • 适合关注中文社交媒体翻译、语言模型风格理解的研究者使用。

中文社交文本中快速演变的俚语、新词和高度风格化的表达,给机器翻译(MT)评估带来巨大挑战。主要障碍包括:(1)数据稀缺——高质量双语平行数据需熟悉平台俚语及语言风格的双语标注员;(2)评估指标局限——传统指标如COMET难以捕捉风格一致性和非标准表达。为此,我们提出CSM-MTBench基准,涵盖五种中-外语言方向,包含两个专家标注子集:Fun Posts(上下文丰富、俚语密集)和Social Snippets(简洁、情绪与风格驱动)。针对不同子集,我们提出定制化评估方法:在Fun Posts中衡量俚语与新词的翻译成功率,在Social Snippets中结合嵌入式指标与大模型评判,评估语气与风格保留程度。对超过20个模型的实验显示,当前MT系统在语义准确性和社交风格处理上表现差异显著。该基准为提升真实中文社交文本翻译能力提供了严谨测试平台。

原文摘要 · Abstract (English)

The prevalence of rapidly evolving slang, neologisms, and highly stylized expressions in informal user-generated text, particularly on Chinese social media, poses significant challenges for Machine Translation (MT) benchmarking. Specifically, we identify two primary obstacles: (1) data scarcity, as high-quality parallel data requires bilingual annotators familiar with platform-specific slang, and stylistic cues in both languages; and (2) metric limitations, where traditional evaluators like COMET often fail to capture stylistic fidelity and nonstandard expressions. To bridge these gaps, we introduce CSM-MTBench, a benchmark covering five Chinese-foreign language directions and consisting of two expert-curated subsets: Fun Posts, featuring context-rich, slang- and neologism-heavy content, and Social Snippets, emphasizing concise, emotion- and style- driven expressions. Furthermore, we propose tailored evaluation approaches for each subset: measuring the translation success rate of slang and neologisms in Fun Posts, while assessing tone and style preservation in Social Snippets via a hybrid of embedding-based metrics and LLM-as-a-judge. Experiments on over 20 models reveal substantial variation in how current MT systems handle semantic fidelity and informal, social-media-specific stylistic cues. CSM-MTBench thus serves as a rigorous testbed for advancing MT systems capable of mastering real-world Chinese social media texts.

机器翻译社交文本风格保持中文俚语

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。