用多级抖动提升视觉模型抗攻击能力，不损语义且效果优于现有方法。
Dithering Defense: Adversarial Robustness of Vision Foundation Models via Multi-Level Floyd-Steinberg Dithering
- 采用多级Floyd-Steinberg抖动作为轻量输入变换，干扰对抗扰动。
- 在六类任务中，中间量化级别结合模糊后处理，性能超越或匹配所有基线。
- 适用于冻结骨干模型的场景，适合关注鲁棒性但不想重训练的研究者。
视觉基础模型广泛用作下游任务的冻结主干，但在对抗攻击下易成单一故障点。本文研究多级Floyd-Steinberg误差扩散抖动作为一种轻量、模型无关的输入变换，可在保留语义内容的同时破坏对抗扰动。不同于以往仅限于二值抖动、灰度CIFAR-10及单个小模型从头训练的工作，本研究在六项任务（分类、分割、深度估计、检索、图像描述、视觉问答）上，使用两种模型族（DINOv2、PaliGemma），以及三种强度递增的攻击方式（PGD、MI-FGSM、SIA）和一个使用直通估计器的自适应攻击进行评估。结果表明，在中间量化级别下应用Floyd-Steinberg抖动，尤其是结合后处理模糊，其表现超过或匹配所有测试基线，包括基于扩散的去噪方法，且对干净输入的退化显著更小。
原文摘要 · Abstract (English)
Vision foundation models are widely used as frozen backbones across many downstream tasks, making them a single point of failure under adversarial attack. We study multi-level Floyd-Steinberg error-diffusion dithering as a lightweight, model-agnostic input transformation that disrupts adversarial perturbations while preserving semantic content. Unlike prior work, which was limited to binary dithering, grayscale CIFAR-10, and a single small model trained from scratch, we evaluate across six tasks (classification, segmentation, depth estimation, retrieval, captioning, visual question answering), two model families (DINOv2, PaliGemma), and three attacks of increasing strength (PGD, MI-FGSM, SIA), as well as an adaptive attacker using a straight-through estimator. Our results show that Floyd-Steinberg dithering at intermediate quantization levels, especially when combined with post-processing blur, exceeds or matches all tested baselines, including diffusion-based denoising, with substantially less degradation on clean inputs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。