只需文字描述就能生成无法被模型学习的保护数据,解决隐私泄露难题。
T2UE: Generating Unlearnable Examples from Text Descriptions
- 用文本描述通过文生图模型生成不可学习噪声,无需原始图像。
- 保护后数据使主流模型跨模态检索性能下降超过60%。
- 适合想在不暴露数据情况下保护隐私的用户和开发者。
大规模预训练框架如CLIP虽推动多模态学习发展,但其依赖网络爬取数据,常包含用户私密信息,引发滥用担忧。无学习样本(UEs)作为应对措施,通过精心设计的不可学习噪声破坏受保护数据的有效表征学习。现有方法需联合优化图像与对应文本的噪声,计算开销大,难以本地执行,迫使用户依赖第三方服务,形成隐私悖论:为保护数据反而需先暴露数据。为此,我们提出文本到不可学习样本(T2UE)框架,仅凭文本描述即可生成UE。T2UE利用文生图(T2I)模型将文本映射至图像空间,并结合误差最小化机制生成有效不可学习噪声。实验表明,经T2UE保护的数据显著降低下游任务(如跨模态检索)中先进模型的表现;该保护效果在多种架构甚至监督学习场景中均具泛化性。本工作证明了‘零接触数据保护’的可行性——仅凭文本描述即可实现数据防护,无需直接暴露原始数据。
原文摘要 · Abstract (English)
Large-scale pre-training frameworks like CLIP have revolutionized multimodal learning, but their reliance on web-scraped datasets, frequently containing private user data, raises serious concerns about misuse. Unlearnable Examples (UEs) have emerged as a promising countermeasure against unauthorized model training, employing carefully crafted unlearnable noise to disrupt the learning of meaningful representations from protected data. Current approaches typically generate UEs by jointly optimizing unlearnable noise for both images and their associated text descriptions (or labels). However, this optimization process is often computationally prohibitive for on-device execution, forcing reliance on external third-party services. This creates a fundamental privacy paradox: users must initially expose their data to these very services to achieve protection, thereby compromising privacy in the process. Such a contradiction has severely hindered the development of practical, scalable data protection solutions. To resolve this paradox, we introduce \textbf{Text-to-Unlearnable Example (T2UE)}, a novel framework that enables users to generate UEs using only text descriptions. T2UE circumvents the need for original image data by employing a text-to-image (T2I) model to map text descriptions into the image (noise) space, combined with an error-minimization framework to produce effective unlearnable noise. Extensive experiments show that T2UE-protected data substantially degrades performance in downstream tasks (e.g., cross-modal retrieval) for state-of-the-art models. Notably, the protective effect generalizes across diverse architectures and even to supervised learning settings. Our work demonstrates the feasibility of "zero-contact data protection", where personal data can be safeguarded based solely on their textual descriptions, eliminating the need for direct data exposure.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。