用GPT-4o+主动提示调优,高效分类小鼠脑组织显微图像。
Active Prompt Tuning Enables Gpt-40 To Do Efficient Classification Of Microscopy Images
- 用GPT-4o结合主动提示调优,减少对标注数据和专家时间的依赖。
- 仅用6只小鼠的提示样本,实现12只中11只正确分类(92%准确率)。
- 相比传统CNN方法效率提升96%,适合生物医学图像快速分析。
传统深度学习方法分类显微图像中的细胞特征需大量时间和人工标注。本文基于GPT-4(V)模型在初步数据集(11只小鼠的Iba-1免疫染色切片)上已验证有效。本研究扩展至更大规模的显微图像数据集,使用更快速的GPT-4o模型与优化提示策略。数据来自18只小鼠(9只Lurcher突变体、9只野生型)的低倍(10x)克里西尔紫染色小脑切片。通过6只小鼠的提示样本,成功将剩余12只小鼠图像正确分类为对照组或突变组,准确率达92%。相较基准的CNN集成模型,效率提升96%,显著降低图像需求和专家工作量。结果表明该方法在不同脑区、不同放大倍数下均具有效性,且开销极小。
原文摘要 · Abstract (English)
Traditional deep learning-based methods for classifying cellular features in microscopy images require time- and labor-intensive processes for training models. Among the current limitations are major time commitments from domain experts for accurate ground truth preparation; and the need for a large amount of input image data. We previously proposed a solution that overcomes these challenges using OpenAI's GPT-4(V) model on a pilot dataset (Iba-1 immuno-stained tissue sections from 11 mouse brains). Results on the pilot dataset were equivalent in accuracy and with a substantial improvement in throughput efficiency compared to the baseline using a traditional Convolutional Neural Net (CNN)-based approach. The present study builds upon this framework using a second unique and substantially larger dataset of microscopy images. Our current approach uses a newer and faster model, GPT-4o, along with improved prompts. It was evaluated on a microscopy image dataset captured at low (10x) magnification from cresyl-violet-stained sections through the cerebellum of a total of 18 mouse brains (9 Lurcher mice, 9 wild-type controls). We used our approach to classify these images either as a control group or Lurcher mutant. Using 6 mice in the prompt set the results were correct classification for 11 out of the 12 mice (92%) with 96% higher efficiency, reduced image requirements, and lower demands on time and effort of domain experts compared to the baseline method (snapshot ensemble of CNN models). These results confirm that our approach is effective across multiple datasets from different brain regions and magnifications, with minimal overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。