arXiv:2601.14994cs.CLcs.AI2026-01被引 1

翻译让数据泄露藏得更深,多语言模型评估需新方法

Obscuring Data Contamination Through Translation: Evidence from Arabic Corpora

  • 用阿拉伯语数据微调模型,发现翻译掩盖了传统污染检测信号
  • 污染程度越高,跨语言答案一致性越强,Min-K%分数持续上升
  • 提出跨语言检测法,能发现英文方法漏掉的隐藏污染

数据污染会削弱大语言模型评估的有效性,使模型依赖记忆而非真正泛化。现有检测方法主要针对英语基准,对多语言污染理解不足。本文通过在不同比例的阿拉伯语数据上微调多个开源大模型,并在原始英语基准上评估,研究多语言环境下的污染动态。为检测记忆行为,扩展了测试槽位猜测方法,引入选项重排策略和Min-K%概率分析,捕捉行为与分布双重污染信号。结果表明:阿拉伯语翻译会抑制传统污染指标,但模型仍从污染数据中获益,尤其具备较强阿拉伯语能力的模型。污染程度上升时,模型的Min-K%分数持续升高,跨语言答案一致性增强。为此,提出翻译感知污染检测方法,通过比较多个翻译版本的信号而非仅依赖英文,可有效识别英文方法遗漏的污染。研究强调需建立多语言、翻译感知的评估体系,以实现公平、透明、可复现的LLM评估。

原文摘要 · Abstract (English)

Data contamination undermines the validity of Large Language Model evaluation by enabling models to rely on memorized benchmark content rather than true generalization. While prior work has proposed contamination detection methods, these approaches are largely limited to English benchmarks, leaving multilingual contamination poorly understood. In this work, we investigate contamination dynamics in multilingual settings by fine-tuning several open-weight LLMs on varying proportions of Arabic datasets and evaluating them on original English benchmarks. To detect memorization, we extend the Tested Slot Guessing method with a choice-reordering strategy and incorporate Min-K% probability analysis, capturing both behavioral and distributional contamination signals. Our results show that translation into Arabic suppresses conventional contamination indicators, yet models still benefit from exposure to contaminated data, particularly those with stronger Arabic capabilities. This effect is consistently reflected in rising Mink% scores and increased cross-lingual answer consistency as contamination levels grow. To address this blind spot, we propose Translation-Aware Contamination Detection, which identifies contamination by comparing signals across multiple translated benchmark variants rather than English alone. The Translation-Aware Contamination Detection reliably exposes contamination even when English-only methods fail. Together, our findings highlight the need for multilingual, translation-aware evaluation pipelines to ensure fair, transparent, and reproducible assessment of LLMs.

大模型评估数据污染多语言翻译检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。