arXiv:2409.19877cs.CLcs.AI2024-09EMNLP被引 1

通过动态抑制重复词元,提升机器翻译的流畅性与多样性。

Contrastive Token Learning with Similarity Decay for Repetition Suppression in Machine Translation

论文配图:Contrastive Token Learning with Similarity Decay for Repetition Suppression in Machine Translation
图 1 · 摘自论文原文
  • 基于注意力权重与词元距离,动态调节重复词元的抑制强度。
  • 在电商文本上显著降低重复率,翻译精度提升12.3%。
  • 已落地阿里全球8个站点,有效提升用户转化与点击率。

跨语言对话与贸易中,神经机器翻译(NMT)虽关键,但生成内容常出现单调与重复问题。传统方法依赖惩罚冗余或词元重现,对长文本及天然冗余的电商描述效果有限,即便使用大语言模型亦然。本文从信息熵角度分析重复成因,认为源于输入文本内不确定性升高。为此提出对比词元学习与相似度衰减(CTSD)算法,依据注意力权重和词元间距动态调节抑制策略。同时构建包含在线商品标题的电商数据集,用于评估易幻觉翻译的鲁棒性。大量实验表明,CTSD在精度与泛化性上显著优于现有方法。线上A/B测试验证其价值:用户参与度与转化率明显提升。该方法已在阿里巴巴全球最大的B2B电商平台alibaba.com的8个多语种站点全流量部署。

原文摘要 · Abstract (English)

For crosslingual conversation and trade, Neural Machine Translation (NMT) is pivotal yet faces persistent challenges with monotony and repetition in generated content. Traditional solutions that rely on penalizing text redundancy or token reoccurrence have shown limited efficacy, particularly for lengthy article and e-commerce descriptions with inherent redundancy, even with the advent of Large Language Models (LLMs). This paper investigates the underlying causes of textual repetition through the lens of information entropy, attributing the phenomenon to the elevated uncertainty within the input text. To address this, a novel algorithm named Contrastive Token Learning with Similarity Decay (CTSD) is introduced, which modulates the suppression of tokens dynamically, informed by varying attention weights and inter-token distances. Furthermore, an e-commerce dataset comprised of title texts of online real items is compiled and released susceptible to hallucination translations to benchmark the algorithm. Extensive evaluations demonstrate that CTSD significantly outperforms existing approaches in precision and generalizability. Additional online A/B testing underscores its practical value, showing marked improvements in user engagement and conversion. Notably, this method has been implemented with full traffic on eight multilingual sites of alibaba.com, the largest B2B e-commerce platform in the world.

机器翻译重复抑制对比学习电商应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。