检验大模型数据污染检测的有效性，发现现有方法在真实场景中表现不佳。
Does Data Contamination Detection Work (Well) for LLMs? A Survey and Evaluation on Detection Assumptions
- 系统梳理50篇论文，归纳出8类检测假设
- 实验证明3种主流方法在预训练数据上效果接近随机猜测
- 提示当前大模型更学分布而非记忆实例，适合评估者参考
大型语言模型（LLMs）在各类基准测试中表现出色，展现出通用任务求解的潜力。然而，由于通常在海量数据上训练，评估中的数据污染问题——即训练数据与评测集存在重叠，导致性能被夸大——成为重要关切。已有多种方法用于识别数据污染，但这些方法依赖特定假设，而这些假设在不同场景下未必成立。为此，我们系统回顾了50篇关于数据污染检测的论文，分类整理其底层假设，并评估其严谨性。我们识别并分析了八类假设，选取其中三类作为案例研究。研究聚焦于直接的、实例级别的数据污染检测，即所谓成员推理攻击（MIA）。分析显示，在大模型预训练数据集上，基于这三种假设的MIA方法表现与随机猜测相当，表明当前大模型可能学习的是数据分布而非记忆单个实例。同时，当已见与未见实例间存在数据分布偏移时，MIA极易失效。
原文摘要 · Abstract (English)
Large language models (LLMs) have demonstrated great performance across various benchmarks, showing potential as general-purpose task solvers. However, as LLMs are typically trained on vast amounts of data, a significant concern in their evaluation is data contamination, where overlap between training data and evaluation datasets inflates performance assessments. Multiple approaches have been developed to identify data contamination. These approaches rely on specific assumptions that may not hold universally across different settings. To bridge this gap, we systematically review 50 papers on data contamination detection, categorize the underlying assumptions, and assess whether they have been rigorously validated. We identify and analyze eight categories of assumptions and test three of them as case studies. Our case studies focus on detecting direct, instance-level data contamination, which is also referred to as Membership Inference Attacks (MIA). Our analysis reveals that MIA approaches based on these three assumptions can have similar performance to random guessing, on datasets used in LLM pretraining, suggesting that current LLMs might learn data distributions rather than memorizing individual instances. Meanwhile, MIA can easily fail when there are data distribution shifts between the seen and unseen instances.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。