arXiv:2602.09611cs.CVcs.AI2026-02KDD被引 1

动态注意力引导水印技术,让视觉语言模型生成更自然的带水印内容。

AGMark: Attention-Guided Dynamic Watermarking for Large Vision-Language Models

  • 根据注意力权重动态识别视觉关键信息,自适应分配水印位置。
  • 结合熵与权重密度,避免无关水印,提升生成质量与视觉一致性。
  • 水印检测准确率超99.36%,抗攻击能力强,适合真实场景部署。

水印已成为大型视觉语言模型(LVLMs)内容溯源与知识产权保护的关键手段。然而,现有视觉无关水印会引入视觉无关标记,破坏视觉语义对齐;而静态视觉特异性水印依赖一次性的权重估计,忽视权重分布密度,无法响应生成过程中视觉依赖的动态变化,可能在长尾阶段引入低质标记。为此,本文提出注意力引导的动态水印框架AGMark:在每一步解码中,基于注意力权重和上下文一致线索动态识别视觉相关证据,构建更自适应的证据权重分布;同时结合不确定性感知(标记熵)与证据校准(权重密度),实现可靠的自适应词汇划分,有效避免无关标记。实验表明,AGMark显著优于传统方法,尤其在生成后期大幅提升视觉语义保真度;在保持推理效率的同时,检测性能达至少99.36% AUC,抗攻击能力达至少88.61% AUC,为高保真多模态水印迈出关键一步。

原文摘要 · Abstract (English)

Watermarking has emerged as a pivotal solution for content traceability and intellectual property protection in large vision language models (LVLMs). However, vision-agnostic watermarks may introduce visually irrelevant tokens and disrupt visual grounding by enforcing indiscriminate pseudo-random biases. Additionally, current vision-specific watermarks rely on a static, one-time estimation of vision-critical weights and ignore the weight distribution density when determining the proportion of protected tokens. This design fails to account for dynamic changes in visual dependence during generation and may introduce low-quality tokens in the long tail. To address these challenges, we propose Attention-Guided Dynamic Watermarking (AGMark), a novel framework that embeds detectable signals while largely preserving visual-semantic fidelity. At each decoding step, AGMark first dynamically identifies semantic-critical evidence based on attention weights for visual relevance, together with context-aware coherence cues, resulting in a more adaptive and well-calibrated evidence-weight distribution. It then determines the proportion of semantic-critical tokens by jointly considering uncertainty awareness (token entropy) and evidence calibration (weight density), thereby enabling more reliable adaptive vocabulary partitioning to avoid irrelevant tokens. Empirical results consistently confirm that AGMark outperforms conventional methods, substantially improving generation quality and yielding particularly strong gains in visual semantic fidelity in the later stages of generation. Our framework maintains highly competitive detection performance (at least 99.36% AUC) and robust attack resilience (at least 88.61% AUC) without sacrificing inference efficiency, taking a significant step toward reliability-preserving multimodal watermarking.

多模态水印视觉语言模型动态生成注意力机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。