arXiv:2603.14254cs.CVcs.LG2026-03被引 1

无需反向传播的测试时自适应方法,让边缘设备也能高效部署模型。

ZOTTA: Test-Time Adaptation with Gradient-Free Zeroth-Order Optimization

  • 用零阶优化仅通过前向传播实现模型自适应,避免反向传播开销。
  • 在ImageNet-C上比SAR方法提升3.9%准确率,内存占用降低84%。
  • 适合资源受限的边缘设备,尤其适用于量化等不可导模型。

测试时自适应(TTA)旨在通过适应无标签测试数据来提升模型在分布偏移下的鲁棒性,但现有方法大多依赖反向传播(BP),计算成本高且不兼容如量化模型等不可导模型,限制了在众多边缘设备上的实际部署。近期无BP方法虽降低了开销,但仍存在架构依赖或优化能力不足的问题。本文提出ZOTTA,一种完全无反向传播的TTA框架,利用零阶优化(ZOO)仅通过前向传播实现高效适应。尽管ZOO理论上可行,但直接应用在高维参数空间中收敛慢且优化不稳定。ZOTTA通过两项机制克服此问题:1)分布鲁棒层选择,自动识别并冻结已提取分布不变特征的层,仅更新域敏感层以降低优化维度并加速收敛;2)空间特征聚合对齐,通过源与目标间全局聚合空间特征对齐,降低梯度方差,稳定优化过程。二者结合实现架构无关且稳定的无BP自适应。在ImageNet-C/R/Sketch/A上的大量实验表明,ZOTTA性能优于或匹配基于BP的方法,例如在ImageNet-C上相比SAR减少84%内存使用并提升3.9%准确率。

原文摘要 · Abstract (English)

Test-time adaptation (TTA) aims to improve model robustness under distribution shifts by adapting to unlabeled test data, but most existing methods rely on backpropagation (BP), which is computationally costly and incompatible with non-differentiable models such as quantized models, limiting practical deployment on numerous edge devices. Recent BP-free approaches alleviate overhead but remain either architecture-specific or limited in optimization capacity to handle high-dimensional models. We propose ZOTTA, a fully BP-free TTA framework that performs efficient adaptation using only forward passes via Zeroth-Order Optimization (ZOO). While ZOO is theoretically appealing, naive application leads to slow convergence under high-dimensional parameter spaces and unstable optimization due to the lack of labels. ZOTTA overcomes these challenges through 1) Distribution-Robust Layer Selection, which automatically identifies and freezes layers that already extract distribution-invariant features, updating only domain-sensitive layers to reduce the optimization dimensionality and accelerate convergence; 2) Spatial Feature Aggregation Alignment, which stabilizes ZOO by aligning globally aggregated spatial features between source and target to reduce gradient variance. Together, these components enable architecture-agnostic and stable BP-free adaptation. Extensive experiments on ImageNet-C/R/Sketch/A show that ZOTTA outperforms or matches BP-based methods, e.g., it reduces memory usage by 84% and improves accuracy by 3.9% over SAR on ImageNet-C. Code:https://github.com/Zhang-Ronghao/zotta.

测试时适应零阶优化边缘部署无反向传播

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。