用进化算法选关键层微调，提升迁移学习效率
Transfer learning optimization based on evolutionary selective fine tuning
- 用进化算法自动筛选需微调的网络层
- 在9个图像数据集上准确率优于或媲美现有方法
- 减少可训练参数，适合资源受限场景
深度学习在图像分析中取得显著进展，但大型全模型训练仍面临计算开销问题。迁移学习通过调整预训练模型适应新任务提供解决方案。传统微调通常更新所有参数，易导致过拟合且计算成本高。本文提出BioTune，一种基于进化算法的自适应微调技术，能选择性地微调关键网络层，以提升迁移学习效率。在涵盖多个领域的9个图像分类数据集上的评估显示，BioTune在准确率和效率方面均优于或媲美AutoRGN、LoRA等现有方法。通过聚焦于相关层的微调，该方法显著降低可训练参数数量，从而减少计算成本，实现对多样数据分布的高效迁移。
原文摘要 · Abstract (English)
Deep learning has shown substantial progress in image analysis. However, the computational demands of large, fully trained models remain a consideration. Transfer learning offers a strategy for adapting pre-trained models to new tasks. Traditional fine-tuning often involves updating all model parameters, which can potentially lead to overfitting and higher computational costs. This paper introduces BioTune, an evolutionary adaptive fine-tuning technique that selectively fine-tunes layers to enhance transfer learning efficiency. BioTune employs an evolutionary algorithm to identify a focused set of layers for fine-tuning, aiming to optimize model performance on a given target task. Evaluation across nine image classification datasets from various domains indicates that BioTune achieves competitive or improved accuracy and efficiency compared to existing fine-tuning methods such as AutoRGN and LoRA. By concentrating the fine-tuning process on a subset of relevant layers, BioTune reduces the number of trainable parameters, potentially leading to decreased computational cost and facilitating more efficient transfer learning across diverse data characteristics and distributions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。