arXiv:2508.02180cs.CV2025-08被引 8

为量化模型设计无需反向传播的实时自适应方法,提升动态环境下的鲁棒性。

Test-Time Model Adaptation for Quantized Neural Networks

  • 仅需两次前向传播实现模型自适应,避免梯度失效问题
  • 在ImageNet-C上使量化ViT-B模型准确率提升5.0%(相比最优方法)
  • 轻量级知识管理机制支持长期持续学习,内存开销极低

部署前对深度模型进行量化是加速推理的常用手段,广泛应用于自动驾驶等实时场景。然而,量化模型在存在领域偏移的动态环境中性能下降更严重,远超全精度模型,理论与实证均证实此现象。现有测试时自适应(TTA)方法因依赖反向传播,难以用于量化模型——因其梯度消失及内存与延迟限制。本文提出一种连续零阶自适应(ZOA)框架,仅通过两次前向传播实现高效适应,消除计算负担。同时设计领域知识管理机制,以极低内存消耗存储并复用不同领域知识,减少干扰,促进长期知识积累。在三种经典架构(包括量化版Transformer与CNN)上的实验表明,该方法显著优于现有方案。在量化W6A6 ViT-B模型上,于ImageNet-C数据集上相较最优方法提升5.0%准确率。代码已开源:https://github.com/DengZeshuai/ZOA。

原文摘要 · Abstract (English)

Quantizing deep models prior to deployment is a widely adopted technique to speed up inference for various real-time applications, such as autonomous driving. However, quantized models often suffer from severe performance degradation in dynamic environments with potential domain shifts and this degradation is significantly more pronounced compared with their full-precision counterparts, as shown by our theoretical and empirical illustrations. To address the domain shift problem, test-time adaptation (TTA) has emerged as an effective solution by enabling models to learn adaptively from test data. Unfortunately, existing TTA methods are often impractical for quantized models as they typically rely on gradient backpropagation--an operation that is unsupported on quantized models due to vanishing gradients, as well as memory and latency constraints. In this paper, we focus on TTA for quantized models to improve their robustness and generalization ability efficiently. We propose a continual zeroth-order adaptation (ZOA) framework that enables efficient model adaptation using only two forward passes, eliminating the computational burden of existing methods. Moreover, we propose a domain knowledge management scheme to store and reuse different domain knowledge with negligible memory consumption, reducing the interference of different domain knowledge and fostering the knowledge accumulation during long-term adaptation. Experimental results on three classical architectures, including quantized transformer-based and CNN-based models, demonstrate the superiority of our methods for quantized model adaptation. On the quantized W6A6 ViT-B model, our ZOA is able to achieve a 5.0\% improvement over the state-of-the-art FOA on ImageNet-C dataset. The source code is available at https://github.com/DengZeshuai/ZOA.

量化自适应零阶优化模型鲁棒性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。