用梯度优化生成更难翻译的文本，提升评测区分度。
Augmenting Text to Increase Translation Difficulty
- 通过可微分难度估计器与对抗优化迭代替换词元。
- 使翻译质量从0.93降至0.82，显著提升难度。
- 无需大模型提示或人工标注，适合模型对比评测。
随着先进机器翻译模型在标准基准上趋于饱和，领域亟需更具挑战性的评估方法以区分不同模型性能。本文提出对抗性翻译优化（ATO），结合对抗优化与可微分翻译难度估计器，通过梯度驱动迭代替换词元，将优化转化为树搜索问题，并采用束搜索求解。该方法不依赖大语言模型提示、昂贵的人工标注或特定任务模型训练，提供了一种基于梯度的替代方案。ATO修改后的基准使平均翻译质量（xCOMET）从0.93降至0.82，优于改写方法（0.88）和零样本基线（0.86）。人工评估显示，修改后文本虽略逊自然，但仍保持合理语法与可读性，且翻译难度显著提升。我们发布了两个各含350条英文文本的数据集及全部代码。
原文摘要 · Abstract (English)
As state-of-the-art machine translation models saturate standard benchmarks, the field needs more challenging evaluations to distinguish between models of varying quality. We propose augmenting existing benchmarks to increase translation difficulty by combining adversarial optimization with a differentiable translation difficulty estimator. Our Adversarial Translation Optimization (ATO) uses gradients from a combined difficulty and fluency objective to iteratively replace tokens. Because each step branches over candidate substitutions at every position, optimization becomes a tree search problem, which we address with Beam Search. ATO offers a gradient-based alternative to LLM-based dataset creation without LLM prompting, expensive human curation, or task-specific model training. Our ATO-modified benchmark lowers average translation quality (xCOMET) from 0.93 to 0.82, compared to 0.88 for paraphrasing and 0.86 for a zero-shot baseline. Human evaluation shows the modified texts are somewhat less natural than the baselines but remain reasonably grammatical and plausible while being substantially harder to translate. We release two datasets of 350 English texts each, generated by our methods, as well as the code.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。