用小参数微调大模型,高效识别多时相珊瑚礁状态
Multi-label classification for multi-temporal, multi-spatial coral reef condition monitoring using vision foundation model with adapter learning
- 用DINOv2+LoRA小规模微调,降低计算开销
- 多时相多地点数据下准确率达64.77%,优于传统模型
- 适合生态监测、环保机构做低成本智能巡护
珊瑚礁生态系统提供关键生态服务,但面临气候变化和人类活动的严重威胁。尽管深度学习已实现珊瑚礁状态的自动分类,传统深度模型在处理复杂水下生态图像时性能有限。视觉基础模型具备高精度和跨域泛化能力,但微调需大量算力且碳排放高。本文采用低秩适配器(LoRA)方法,将DINOv2视觉基础模型与LoRA结合,利用泰国涛岛15个潜水点的多时相水下调查图像进行训练,所有图像均按公民科学保护项目通用标准标注。实验表明,DINOv2-LoRA模型匹配率高达64.77%,优于最佳传统模型的60.34%;同时可训练参数从1,100M降至5.91M。不同季节与站点的迁移学习实验显示该模型具有优异泛化能力。本研究首次探索了基础模型在多时相、多空间珊瑚礁多标签分类中的高效适应,为珊瑚礁监测、保护与管理提供了新工具。
原文摘要 · Abstract (English)
Coral reef ecosystems provide essential ecosystem services, but face significant threats from climate change and human activities. Although advances in deep learning have enabled automatic classification of coral reef conditions, conventional deep models struggle to achieve high performance when processing complex underwater ecological images. Vision foundation models, known for their high accuracy and cross-domain generalizability, offer promising solutions. However, fine-tuning these models requires substantial computational resources and results in high carbon emissions. To address these challenges, adapter learning methods such as Low-Rank Adaptation (LoRA) have emerged as a solution. This study introduces an approach integrating the DINOv2 vision foundation model with the LoRA fine-tuning method. The approach leverages multi-temporal field images collected through underwater surveys at 15 dive sites at Koh Tao, Thailand, with all images labeled according to universal standards used in citizen science-based conservation programs. The experimental results demonstrate that the DINOv2-LoRA model achieved superior accuracy, with a match ratio of 64.77%, compared to 60.34% achieved by the best conventional model. Furthermore, incorporating LoRA reduced the trainable parameters from 1,100M to 5.91M. Transfer learning experiments conducted under different temporal and spatial settings highlight the exceptional generalizability of DINOv2-LoRA across different seasons and sites. This study is the first to explore the efficient adaptation of foundation models for multi-label classification of coral reef conditions under multi-temporal and multi-spatial settings. The proposed method advances the classification of coral reef conditions and provides a tool for monitoring, conserving, and managing coral reef ecosystems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。