提出对抗视角下的数据集审计评估框架,揭示现有方法易被攻击漏洞。
DATABench: Evaluating Dataset Auditing in Deep Learning from an Adversarial Perspective
- 按内/外特征分类现有审计方法,构建对抗攻击新范式。
- 设计17种隐蔽攻击与5种伪造攻击,测试审计系统鲁棒性。
- 发现所有审计方法在对抗环境下均不可靠,亟需更安全方案。
深度学习广泛应用依赖于训练数据集的质量与构成,但其使用情况常不公开,引发隐私与版权担忧。数据集审计技术旨在判断特定数据集是否被用于训练可疑模型,是解决透明度缺口的潜在方案。然而,现有方法对专门设计的对抗攻击的鲁棒性仍缺乏研究。本文首次从对抗视角系统评估数据集审计,提出新分类体系:基于内部特征(IF)与外部特征(EF)。定义两类攻击:逃避攻击(隐藏数据集使用)与伪造攻击(错误指认未用数据集)。据此提出系统化攻击策略:解耦、移除、检测用于逃避;基于对抗样本的方法用于伪造。由此构建新基准DATABench,包含17种逃避攻击、5种伪造攻击及9种代表性审计方法。大规模实验表明,所有评测审计方法在对抗环境下均缺乏足够鲁棒性或区分性。结果凸显开发能抵御复杂对抗操纵的更安全、可靠审计方法的紧迫性。代码已开源:https://github.com/shaoshuo-ss/DATABench。
原文摘要 · Abstract (English)
The widespread application of Deep Learning across diverse domains hinges critically on the quality and composition of training datasets. However, the common lack of disclosure regarding their usage raises significant privacy and copyright concerns. Dataset auditing techniques, which aim to determine if a specific dataset was used to train a given suspicious model, provide promising solutions to addressing these transparency gaps. While prior work has developed various auditing methods, their resilience against dedicated adversarial attacks remains largely unexplored. To bridge the gap, this paper initiates a comprehensive study evaluating dataset auditing from an adversarial perspective. We start with introducing a novel taxonomy, classifying existing methods based on their reliance on internal features (IF) (inherent to the data) versus external features (EF) (artificially introduced for auditing). Subsequently, we formulate two primary attack types: evasion attacks, designed to conceal the use of a dataset, and forgery attacks, intending to falsely implicate an unused dataset. Building on the understanding of existing methods and attack objectives, we further propose systematic attack strategies: decoupling, removal, and detection for evasion; adversarial example-based methods for forgery. These formulations and strategies lead to our new benchmark, DATABench, comprising 17 evasion attacks, 5 forgery attacks, and 9 representative auditing methods. Extensive evaluations using DATABench reveal that none of the evaluated auditing methods are sufficiently robust or distinctive under adversarial settings. These findings underscore the urgent need for developing a more secure and reliable dataset auditing method capable of withstanding sophisticated adversarial manipulation. Code is available in https://github.com/shaoshuo-ss/DATABench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。