用多模态大模型实现珊瑚礁智能监测，精准分割与分类。
YH-MINER: Multimodal Intelligent System for Natural Ecological Reef Metric Extraction
- 通过目标检测生成先验框，指导复杂水下场景的像素级分割。
- 在低光和密集遮挡下实现88%的属级分类准确率，提取关键生态指标。
- 系统可扩展性强，适合集成到水下机器人实现全自动监测流程。
珊瑚礁对维持海洋生物多样性和生态过程（如养分循环、生境提供）至关重要，正面临日益严峻的威胁，亟需高效监测手段。珊瑚礁生态监测面临人工分析效率低和复杂水下场景分割精度不足的双重挑战。本研究开发了YH-MINER系统，构建以多模态大模型（MLLM）为核心的智能框架，实现‘目标检测-语义分割-先验输入’一体化流程。系统采用目标检测模块（[email protected]=0.78）生成珊瑚实例的空间先验框，驱动分割模块在低光照和密集遮挡条件下完成像素级分割。分割掩码与微调后的分类指令作为先验输入，输入基于Qwen2-VL的多模态模型，实现88%的属级分类准确率，并同步提取核心生态指标。同时，系统通过标准化接口保持多模态模型的可扩展性，为未来集成至多模态智能体驱动的水下机器人奠定基础，支持从‘图像采集-先验生成-实时分析’全流程自动化。
原文摘要 · Abstract (English)
Coral reefs, crucial for sustaining marine biodiversity and ecological processes (e.g., nutrient cycling, habitat provision), face escalating threats, underscoring the need for efficient monitoring. Coral reef ecological monitoring faces dual challenges of low efficiency in manual analysis and insufficient segmentation accuracy in complex underwater scenarios. This study develops the YH-MINER system, establishing an intelligent framework centered on the Multimodal Large Model (MLLM) for "object detection-semantic segmentation-prior input". The system uses the object detection module ([email protected]=0.78) to generate spatial prior boxes for coral instances, driving the segment module to complete pixel-level segmentation in low-light and densely occluded scenarios. The segmentation masks and finetuned classification instructions are fed into the Qwen2-VL-based multimodal model as prior inputs, achieving a genus-level classification accuracy of 88% and simultaneously extracting core ecological metrics. Meanwhile, the system retains the scalability of the multimodal model through standardized interfaces, laying a foundation for future integration into multimodal agent-based underwater robots and supporting the full-process automation of "image acquisition-prior generation-real-time analysis".
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。