通过提问检测大模型训练数据是否含版权内容，无需访问原始数据。
DIS-CO: Discovering Copyrighted Content in VLMs Training Data
- 用特定画面反复提问模型，从自由文本回复中推断版权内容是否存在。
- 在14,000帧的MovieTection基准上，检测准确率接近前人方法的两倍。
- 发现所有测试模型均可能包含版权内容，适合版权合规研究者使用。
如何在无法获取训练数据的情况下验证大型视觉语言模型（VLM）是否使用了受版权保护的内容？基于模型能识别其训练数据中的图像这一假设，我们提出DIS-CO，一种推断版权内容是否被纳入训练的新方法。通过反复向VLM查询目标版权材料的具体帧，并分析其生成的自由文本补全，可提取出内容身份。为评估效果，我们构建了MovieTection基准，包含14,000帧及其详细描述，素材来自模型训练截止时间前后发布的电影。结果表明，DIS-CO显著提升了检测性能，在有logits可用的模型上，平均AUC近乎翻倍。研究还揭示了一个更广泛的问题：所有测试模型似乎都以某种程度接触过版权内容。代码与数据已公开于https://github.com/avduarte333/DIS-CO。
原文摘要 · Abstract (English)
How can we verify whether copyrighted content was used to train a large vision-language model (VLM) without direct access to its training data? Motivated by the hypothesis that a VLM is able to recognize images from its training corpus, we propose DIS-CO, a novel approach to infer the inclusion of copyrighted content during the model's development. By repeatedly querying a VLM with specific frames from targeted copyrighted material, DIS-CO extracts the content's identity through free-form text completions. To assess its effectiveness, we introduce MovieTection, a benchmark comprising 14,000 frames paired with detailed captions, drawn from films released both before and after a model's training cutoff. Our results show that DIS-CO significantly improves detection performance, nearly doubling the average AUC of the best prior method on models with logits available. Our findings also highlight a broader concern: all tested models appear to have been exposed to some extent to copyrighted content. Our code and data are available at https://github.com/avduarte333/DIS-CO
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。