给视觉大模型嵌入可检测水印,防非法复制
ActiveMark: on watermarking of visual foundation models via massive activations
- 通过微调部分层和编码器-解码器网络,将水印嵌入模型内部表示
- 水印在下游任务微调后仍可检测,误检率和漏检率均很低
- 适合保护视觉大模型版权,防止未经授权的分发
视觉基础模型(VFMs)在大规模数据上训练,可微调用于多种下游任务,在计算机视觉应用中表现出色。由于数据收集与训练成本高昂,部分模型所有者需通过许可证分发模型以保护知识产权。但存在用户非法重分发模型的风险。因此,可靠的版权验证工具至关重要,可用于区分被重分发的受保护模型与独立模型。本文提出一种方法:通过微调少量表达性强的层及小型编码器-解码器网络,将数字水印嵌入一组预留输入图像的内部表征中。重要的是,水印在功能副本(如针对特定下游任务微调后的模型)中仍可检测。理论与实验表明,该方法具有极低的非水印模型误检率和水印模型漏检率。
原文摘要 · Abstract (English)
Being trained on large and vast datasets, visual foundation models (VFMs) can be fine-tuned for diverse downstream tasks, achieving remarkable performance and efficiency in various computer vision applications. The high computation cost of data collection and training motivates the owners of some VFMs to distribute them alongside the license to protect their intellectual property rights. However, a dishonest user of the protected model's copy may illegally redistribute it, for example, to make a profit. As a consequence, the development of reliable ownership verification tools is of great importance today, since such methods can be used to differentiate between a redistributed copy of the protected model and an independent model. In this paper, we propose an approach to ownership verification of visual foundation models by fine-tuning a small set of expressive layers of a VFM along with a small encoder-decoder network to embed digital watermarks into an internal representation of a hold-out set of input images. Importantly, the watermarks embedded remain detectable in the functional copies of the protected model, obtained, for example, by fine-tuning the VFM for a particular downstream task. Theoretically and experimentally, we demonstrate that the proposed method yields a low probability of false detection of a non-watermarked model and a low probability of false misdetection of a watermarked model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。