用通用噪声生成器让图像无法被分割模型学习,保护隐私。
UnSeg: One Universal Unlearnable Example Generator is Enough against All Image Segmentation
- 基于SAM构建通用噪声生成器,通过双层优化实现跨任务泛化。
- 在6类任务、10个数据集上验证,使分割性能大幅下降。
- 适合关注数据隐私与高效防护的研究者使用。
图像分割是将图像像素分组为语义有意义区域的关键视觉任务,对理解真实场景至关重要。然而,大规模图像分割模型在未经授权的私有数据上训练引发日益严重的隐私问题。本文提出一种新型不可学习分割(UnSeg)框架,通过生成并添加不可学习噪声,使图像无法用于模型训练。该框架训练一个通用噪声生成器,可将任意下游图像转化为不可学习版本。噪声生成器基于段落任意模型(SAM)微调,采用双层优化,在交互式分割数据集上最小化同架构但从零训练的代理模型的训练误差。我们在6种主流图像分割任务、10个广泛使用的数据集和7种不同网络架构上实证验证了UnSeg的有效性,结果表明不可学习图像能显著降低分割性能。本工作为高效利用基础模型保护图像免受分割模型侵害提供了新思路。
原文摘要 · Abstract (English)
Image segmentation is a crucial vision task that groups pixels within an image into semantically meaningful segments, which is pivotal in obtaining a fine-grained understanding of real-world scenes. However, an increasing privacy concern exists regarding training large-scale image segmentation models on unauthorized private data. In this work, we exploit the concept of unlearnable examples to make images unusable to model training by generating and adding unlearnable noise into the original images. Particularly, we propose a novel Unlearnable Segmentation (UnSeg) framework to train a universal unlearnable noise generator that is capable of transforming any downstream images into their unlearnable version. The unlearnable noise generator is finetuned from the Segment Anything Model (SAM) via bilevel optimization on an interactive segmentation dataset towards minimizing the training error of a surrogate model that shares the same architecture with SAM but is trained from scratch. We empirically verify the effectiveness of UnSeg across 6 mainstream image segmentation tasks, 10 widely used datasets, and 7 different network architectures, and show that the unlearnable images can reduce the segmentation performance by a large margin. Our work provides useful insights into how to leverage foundation models in a data-efficient and computationally affordable manner to protect images against image segmentation models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。