AI识别野生牡蛎准确率不如人类,但速度更快。
Is AI currently capable of identifying wild oysters? A comparison of human annotators against the AI model, ODYSSEE
- 用深度学习模型ODYSSEE自动识别牡蛎图像中的活体
- 模型准确率63%,低于专家74%和非专家75%
- 图像质量越高,人工越准,模型反而更差
牡蛎在生态与商业上均具重要意义,需频繁监测种群数量(如丰度、生长、死亡率)。当前监测方法多依赖破坏性采样和大量人工,不适用于小规模或敏感环境。近期提出的ODYSSEE模型利用深度学习技术,通过野外拍摄的视频或图像识别活体牡蛎以评估丰度。本研究将该模型与专家及非专家标注者对比,评估其识别有效性,并分析预测误差来源。尽管模型推理速度显著快于专家(39.6秒 vs. 2.34±0.61小时)和非专家(4.50±1.46小时),但其对活体牡蛎的识别准确率仅为63%,低于专家(74%)和非专家(75%)。图像质量是影响模型与人工准确率的关键因素:高质量图像提升人类表现,却降低模型性能。尽管当前精度不足,基于本研究结果,未来在更高清图像、更多活体样本训练及引入额外标注类别后,模型预测能力有望显著提升。后续研究应聚焦提升活体与死体牡蛎的区分能力。
原文摘要 · Abstract (English)
Oysters are ecologically and commercially important species that require frequent monitoring to track population demographics (e.g. abundance, growth, mortality). Current methods of monitoring oyster reefs often require destructive sampling methods and extensive manual effort. Therefore, they are suboptimal for small-scale or sensitive environments. A recent alternative, the ODYSSEE model, was developed to use deep learning techniques to identify live oysters using video or images taken in the field of oyster reefs to assess abundance. The validity of this model in identifying live oysters on a reef was compared to expert and non-expert annotators. In addition, we identified potential sources of prediction error. Although the model can make inferences significantly faster than expert and non-expert annotators (39.6 s, $2.34 \pm 0.61$ h, $4.50 \pm 1.46$ h, respectively), the model overpredicted the number of live oysters, achieving lower accuracy (63\%) in identifying live oysters compared to experts (74\%) and non-experts (75\%) alike. Image quality was an important factor in determining the accuracy of the model and the annotators. Better quality images improved human accuracy and worsened model accuracy. Although ODYSSEE was not sufficiently accurate, we anticipate that future training on higher-quality images, utilizing additional live imagery, and incorporating additional annotation training classes will greatly improve the model's predictive power based on the results of this analysis. Future research should address methods that improve the detection of living vs. dead oysters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。