提出首个针对对比学习模型的数据集所有权验证方法。
Dataset Ownership Verification in Contrastive Pre-trained Models
- 基于嵌入空间中实例关系变化设计验证机制。
- 在多个对比预训练模型上实现显著低于0.05的p值。
- 适合关注数据版权保护的研究者与开发者。
高质量开源数据集的构建需大量人工投入,已成为深度学习快速发展的主要推动力。同时,保护这些数据集对数据拥有者至关重要。数据集所有权验证在此领域显得尤为关键,但现有方法多局限于有监督模型,难以直接应用于日益流行的无监督预训练模型。本文提出首个专为对比学习自监督预训练模型设计的数据集所有权验证方法。其核心目标是判断可疑的黑箱主干网络是否曾在特定未标注数据集上进行过预训练,帮助数据所有者维护权利。该方法基于实证发现:当模型在目标数据集上训练时,嵌入空间中的单例与双例实例关系会显著不同于未使用目标数据集的模型。我们在SimCLR、BYOL、SimSiam、MOCO v3和DINO等多种对比预训练模型上验证了该方法的有效性,结果表明其拒绝原假设的p值远低于0.05,优于所有先前方法。代码已公开于https://github.com/xieyc99/DOV4CL。
原文摘要 · Abstract (English)
High-quality open-source datasets, which necessitate substantial efforts for curation, has become the primary catalyst for the swift progress of deep learning. Concurrently, protecting these datasets is paramount for the well-being of the data owner. Dataset ownership verification emerges as a crucial method in this domain, but existing approaches are often limited to supervised models and cannot be directly extended to increasingly popular unsupervised pre-trained models. In this work, we propose the first dataset ownership verification method tailored specifically for self-supervised pre-trained models by contrastive learning. Its primary objective is to ascertain whether a suspicious black-box backbone has been pre-trained on a specific unlabeled dataset, aiding dataset owners in upholding their rights. The proposed approach is motivated by our empirical insights that when models are trained with the target dataset, the unary and binary instance relationships within the embedding space exhibit significant variations compared to models trained without the target dataset. We validate the efficacy of this approach across multiple contrastive pre-trained models including SimCLR, BYOL, SimSiam, MOCO v3, and DINO. The results demonstrate that our method rejects the null hypothesis with a $p$-value markedly below $0.05$, surpassing all previous methodologies. Our code is available at https://github.com/xieyc99/DOV4CL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。