为自回归图像生成模型设计了更抗干扰的水印方法
ClusterMark: Towards Robust Watermarking for Autoregressive Image Generators with Visual Token Clustering
- 通过视觉标记聚类,将相似标记归入同一组进行水印
- 在图像扰动和重生成攻击下仍保持高可检测性
- 速度快,适合实际部署,尤其适合生成内容溯源
针对潜在扩散模型的生成水印近期展现出良好鲁棒性,但其在自回归(AR)图像生成模型中的应用仍不充分。自回归模型通过逐个预测视觉标记序列,并由VQ-VAE解码为像素。受大语言模型水印方法启发,我们研究基于标记级别的水印方案,即根据前序标记偏置下一个标记预测。发现直接迁移该方法虽可行,但在常见图像扰动下水印可检测性显著下降。为此,我们提出基于视觉标记聚类的水印方法:将相似标记分配至同一组(红或绿)。研究了无需训练的聚类设置,以及结合更精准微调的标记或聚类预测器。实验表明,基于聚类的水印在扰动和重生成攻击下鲁棒性大幅提升,同时保持图像质量,优于多个基线与同期工作。此外,验证速度极快,接近轻量级后处理水印技术。
原文摘要 · Abstract (English)
In-generation watermarking for latent diffusion models has recently shown high robustness in marking generated images for easier detection and attribution. However, its application to autoregressive (AR) image models is underexplored. Autoregressive models generate images by autoregressively predicting a sequence of visual tokens that are then decoded into pixels using a VQ-VAE decoder. Inspired by KGW watermarking for large language models, we examine token-level watermarking schemes that bias the next-token prediction based on prior tokens. We find that a direct transfer of these schemes works in principle, but the detectability of the watermarks decreases considerably under common image perturbations. As a remedy, we propose a watermarking approach based on visual token clustering, which assigns similar tokens to the same set (red or green). We investigate token clustering in a training-free setting, as well as in combination with a more accurate fine-tuned token or cluster predictor. Overall, our experiments show that cluster-based watermarks greatly improve robustness against perturbations and regeneration attacks while preserving image quality, outperforming a set of baselines and concurrent works. Moreover, our methods offer fast verification runtime, comparable to lightweight post-hoc watermarking techniques.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。