通过重置权重恢复模型可塑性，提升迁移学习速度与效果
Restoring Neural Network Plasticity for Faster Transfer Learning
- 针对预训练模型在微调时梯度消失问题，提出定向权重重初始化策略
- 在多个图像分类任务上实现更快收敛，测试准确率显著提升
- 无需额外计算开销，兼容主流迁移学习流程，适合快速适配新数据
基于ImageNet预训练的迁移学习已成为计算机视觉的标准实践。迁移学习指将预训练模型权重在下游任务上进行微调，而该任务通常与ImageNet无关。然而，预训练权重可能趋于饱和，导致梯度微弱，无法有效适应下游任务，这种现象被称为神经可塑性丧失。这会阻碍模型充分适配目标域，尤其当下游数据集特性异常时更为明显。尽管该问题在持续学习中被广泛研究，但在迁移学习中仍相对未受重视。本文提出一种针对性的权重重初始化策略，在微调前恢复神经可塑性。实验表明，卷积神经网络（CNN）和视觉变换器（ViTs）均受益于该方法，在多个图像分类基准上实现更高测试准确率与更快收敛。该方法引入的计算开销可忽略不计，且与常见迁移学习流程完全兼容。
原文摘要 · Abstract (English)
Transfer learning with models pretrained on ImageNet has become a standard practice in computer vision. Transfer learning refers to fine-tuning pretrained weights of a neural network on a downstream task, typically unrelated to ImageNet. However, pretrained weights can become saturated and may yield insignificant gradients, failing to adapt to the downstream task. This hinders the ability of the model to train effectively, and is commonly referred to as loss of neural plasticity. Loss of plasticity may prevent the model from fully adapting to the target domain, especially when the downstream dataset is atypical in nature. While this issue has been widely explored in continual learning, it remains relatively understudied in the context of transfer learning. In this work, we propose the use of a targeted weight re-initialization strategy to restore neural plasticity prior to fine-tuning. Our experiments show that both convolutional neural networks (CNNs) and vision transformers (ViTs) benefit from this approach, yielding higher test accuracy with faster convergence on several image classification benchmarks. Our method introduces negligible computational overhead and is compatible with common transfer learning pipelines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。