用少量胚胎图像和描述，训练出能自动生成发育报告的多模态AI。
InVitroVision: a Multi-Modal AI Model for Automated Description of Embryo Development using Natural Language
- 用1000张胚胎图像+描述微调PaliGemma-2，实现图文理解。
- 在描述准确性上优于商用模型ChatGPT 5.2和基础模型。
- 适合临床辅助决策、少样本医学场景的快速部署。
人工智能在体外受精（IVF）中的应用展现出提升决策一致性与标准化的潜力，但通常依赖标注数据，且未充分利用IVF数据的多模态特性。我们研究了基础视觉语言模型是否可通过微调，预测胚胎形态与发育过程的自然语言描述。基于公开的胚胎时间延时数据集，我们仅用1,000张图像及其对应的胚胎形态、细胞周期和发育阶段描述，对PaliGemma-2模型进行微调，构建了InVitroVision模型。结果表明，该模型在整体性能上优于商业模型ChatGPT 5.2及基础模型，且随着训练数据量增加，性能持续提升。本研究证明，基础视觉语言模型可在有限数据条件下泛化至IVF任务，实现对胚胎形态与发育过程的自然语言描述生成。该方法有助于大语言模型从文献与指南中检索信息与科学证据，并为IVF中多个下游任务的少样本适应提供可能。
原文摘要 · Abstract (English)
The application of artificial intelligence (AI) in IVF has shown promise in improving consistency and standardization of decisions, but often relies on annotated data and does not make use of the multimodal nature of IVF data. We investigated whether foundational vision-language models can be fine-tuned to predict natural language descriptions of embryo morphology and development. Using a publicly available embryo time-lapse dataset, we fine-tuned PaliGemma-2, a multi-modal vision-language model, with only 1,000 images and corresponding captions, describing embryo morphology, embryonic cell cycle and developmental stage. Our results show that the fine-tuned model, InVitroVision, outperformed a commercial model, ChatGPT 5.2, and base models in overall metrics, with performance improving with larger training datasets. This study demonstrates the potential of foundational vision-language models to generalize to IVF tasks with limited data, enabling the prediction of natural language descriptions of embryo morphology and development. This approach may facilitate the use of large language models to retrieve information and scientific evidence from relevant publications and guidelines, and has implications for few-shot adaptation to multiple downstream tasks in IVF.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。