用低秩适配和动态损失权重,高效为扩散模型嵌入水印。
An Efficient Watermarking Method for Latent Diffusion Models via Low-Rank Adaptation and Dynamic Loss Weighting
- 通过引入可训练低秩参数,在不改动原模型权重的前提下嵌入水印。
- 动态调整损失权重,生成图像质量与水印保真度均达到领先水平。
- 适合大规模扩散模型,兼顾效率、质量与跨模型泛化能力。
深度神经网络的快速普及推动了模型水印技术的发展,因为训练好的模型本身是重要知识产权。现有方法主要聚焦于修改模型参数或改变采样行为。然而,随着模型规模增大,提升水印嵌入效率对缓解计算负担至关重要。本论文提出一种基于低秩适配(LoRA)的高效水印方法,用于潜空间扩散模型(LDMs)。核心思想是在冻结的LDM中引入可训练的低秩参数以嵌入水印,从而保持原始模型权重完整。此外,设计了一种动态损失权重调度器,自适应平衡生成质量与水印保真度,使模型在最小影响生成质量的前提下实现有效水印嵌入。实验表明,该方法在嵌入速度与准确性上表现优异,生成图像质量高,鲁棒性在部分场景下甚至优于现有最优方法。且在不同数据集和基础LDM间具有良好泛化能力。代码已开源:https://github.com/MrDongdongLin/EW-LoRA。
原文摘要 · Abstract (English)
The rapid proliferation of Deep Neural Networks (DNNs) is driving a surge in model watermarking technologies, as the trained models themselves constitute valuable intellectual property. Existing watermarking approaches primarily focus on modifying model parameters or altering sampling behaviors. However, with the emergence of increasingly large models, improving the efficiency of watermark embedding becomes essential to manage increasing computational demands. Prioritizing efficiency not only optimizes resource utilization, making the watermarking process more applicable for large models, but also mitigates potential degradation of model performance. In this paper, we propose an efficient watermarking method for Latent Diffusion Models (LDMs) based on Low-Rank Adaptation (LoRA). The core idea is to introduce trainable low-rank parameters into the frozen LDM to embed watermark, thereby preserving the integrity of the original model weights. Furthermore, a dynamic loss weight scheduler is designed to adaptively balance the objectives of generative quality and watermark fidelity, enabling the model to achieve effective watermark embedding with minimal impact on quality of the generated images. Experimental results show that the proposed method ensures fast and accurate watermark embedding and a high quality of the generated images, at the same time maintaining a level of robustness aligned - in some cases superior - with state-of-the-art approaches. Moreover, the method generalizes well across different datasets and base LDMs. Codes are available at: https://github.com/MrDongdongLin/EW-LoRA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。