arXiv:2503.20354cs.CVcs.LG2025-03CVPR被引 15

通过动态稀疏化激活,实现低内存的实时模型自适应。

SURGEON: Memory-Adaptive Fully Test-Time Adaptation via Dynamic Activation Sparsity

  • 根据梯度重要性和内存消耗动态调整各层激活稀疏度。
  • 在多个数据集上实现比基线更低内存占用和更高准确率。
  • 适用于资源受限设备,无需修改训练流程或网络结构。

尽管深度模型已广泛部署于移动终端,但因各种部署干扰导致其精度显著下降。测试时自适应(TTA)通过在线适应未标记目标数据来提升模型性能。然而,基于反向传播的多数TTA方法存在显著内存开销,阻碍了其在资源受限终端的有效部署。为此,我们提出SURGEON,一种在不依赖特定网络架构或修改原始训练流程的前提下,大幅降低全测试时自适应(FTTA)内存开销同时保持相当准确率提升的方法。具体而言,我们提出一种新颖的动态激活稀疏策略,在适应过程中以层特定的动态比率直接剪枝激活值,实现学习能力与内存消耗的数据敏感灵活控制。其中,梯度重要性与层激活内存两个指标分别用于决定层间剪枝比例,反映精度贡献与内存效率。实验表明,该方法不仅显著降低内存使用,还超越基线,在多种数据集、架构和任务上达到最先进性能。

原文摘要 · Abstract (English)

Despite the growing integration of deep models into mobile terminals, the accuracy of these models declines significantly due to various deployment interferences. Test-time adaptation (TTA) has emerged to improve the performance of deep models by adapting them to unlabeled target data online. Yet, the significant memory cost, particularly in resource-constrained terminals, impedes the effective deployment of most backward-propagation-based TTA methods. To tackle memory constraints, we introduce SURGEON, a method that substantially reduces memory cost while preserving comparable accuracy improvements during fully test-time adaptation (FTTA) without relying on specific network architectures or modifications to the original training procedure. Specifically, we propose a novel dynamic activation sparsity strategy that directly prunes activations at layer-specific dynamic ratios during adaptation, allowing for flexible control of learning ability and memory cost in a data-sensitive manner. Among this, two metrics, Gradient Importance and Layer Activation Memory, are considered to determine the layer-wise pruning ratios, reflecting accuracy contribution and memory efficiency, respectively. Experimentally, our method surpasses the baselines by not only reducing memory usage but also achieving superior accuracy, delivering SOTA performance across diverse datasets, architectures, and tasks.

测试时自适应内存优化动态稀疏轻量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。