针对鱼群重叠与高相似性难题,提出实时鱼群追踪新模型。
FMRFT: Fusion Mamba and DETR for Query Time Sequence Intersection Fish Tracking
- 融合Mamba与DETR结构,实现多帧时序记忆与特征提取
- 在自建数据集上达IDF1 90.3%、MOTA 94.3%
- 适合工厂化养殖中复杂场景的实时鱼群追踪
通过深度学习技术进行鱼群追踪可早期发现疾病或饥饿引发的异常行为,对工业水产养殖具有重要意义。然而,水下反光、鱼体高度相似、快速游动及相互遮挡等问题给多目标追踪带来挑战。为此,本文构建了一个复杂多场景鲟鱼追踪数据集,并提出FMRFT模型,一种实时端到端鱼群追踪解决方案。该模型引入低视频内存消耗的Mamba in Mamba(MIM)架构,支持多帧时序记忆与特征提取,有效应对跨帧追踪难题。同时,结合查询时间序列交集(QTSI)模块,利用RT-DETR的优越特征交互与前帧处理能力,精准管理遮挡目标并减少冗余追踪帧。在自建数据集上训练与测试,模型取得IDF1 90.3%和MOTA 94.3%的性能。实验表明,FMRFT能有效应对鱼群高相似性与相互遮挡问题,在工厂化养殖环境中实现高精度稳定追踪。
原文摘要 · Abstract (English)
Early detection of abnormal fish behavior caused by disease or hunger can be achieved through fish tracking using deep learning techniques, which holds significant value for industrial aquaculture. However, underwater reflections and some reasons with fish, such as the high similarity, rapid swimming caused by stimuli and mutual occlusion bring challenges to multi-target tracking of fish. To address these challenges, this paper establishes a complex multi-scenario sturgeon tracking dataset and introduces the FMRFT model, a real-time end-to-end fish tracking solution. The model incorporates the low video memory consumption Mamba In Mamba (MIM) architecture, which facilitates multi-frame temporal memory and feature extraction, thereby addressing the challenges to track multiple fish across frames. Additionally, the FMRFT model with the Query Time Sequence Intersection (QTSI) module effectively manages occluded objects and reduces redundant tracking frames using the superior feature interaction and prior frame processing capabilities of RT-DETR. This combination significantly enhances the accuracy and stability of fish tracking. Trained and tested on the dataset, the model achieves an IDF1 score of 90.3% and a MOTA accuracy of 94.3%. Experimental results show that the proposed FMRFT model effectively addresses the challenges of high similarity and mutual occlusion in fish populations, enabling accurate tracking in factory farming environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。