arXiv:2411.09933cs.CVcs.AI2024-11中稿 · NeurIPS被引 5

用进化优化合并技术,仅用50个日语样本训练出高效医学报告生成模型。

JRadiEvo: A Japanese Radiology Report Generation Model Enhanced by Evolutionary Optimization of Model Merging

  • 通过进化算法优化模型融合,提升非医疗大模型在医学领域的适应性。
  • 仅用50个日语放射科报告样本,生成效果超过依赖大规模数据的模型。
  • 80亿参数的小型模型可本地部署,适合严苛隐私要求的医院环境。

随着大语言模型的快速发展,基础模型在医疗领域应用日益广泛。然而,现有方法通常需要大量标注数据进行微调,且多基于英文数据集,限制了非英语地区医生的应用。为此,我们提出一种基于进化优化模型融合的日本放射科报告生成模型(JRadiEvo),首次将非医疗视觉-语言基础模型通过该方法扩展至医疗领域。仅使用50个公开数据集中的日语翻译样本,即可生成准确的放射科报告,其性能优于许多基于更大规模数据训练的前沿模型。该模型仅含80亿参数,体积小、效率高,可部署于医院本地系统,满足严格隐私与安全要求,为资源受限的医疗场景提供实用解决方案。

原文摘要 · Abstract (English)

With the rapid advancement of large language models (LLMs), foundational models (FMs) have seen significant advancements. Healthcare is one of the most crucial application areas for these FMs, given the significant time and effort required for physicians to analyze large volumes of patient data. Recent efforts have focused on adapting multimodal FMs to the medical domain through techniques like instruction-tuning, leading to the development of medical foundation models (MFMs). However, these approaches typically require large amounts of training data to effectively adapt models to the medical field. Moreover, most existing models are trained on English datasets, limiting their practicality in non-English-speaking regions where healthcare professionals and patients are not always fluent in English. The need for translation introduces additional costs and inefficiencies. To address these challenges, we propose a \textbf{J}apanese \textbf{Radi}ology report generation model enhanced by \textbf{Evo}lutionary optimization of model merging (JRadiEvo). This is the first attempt to extend a non-medical vision-language foundation model to the medical domain through evolutionary optimization of model merging. We successfully created a model that generates accurate Japanese reports from X-ray images using only 50 translated samples from publicly available data. This model, developed with highly efficient use of limited data, outperformed leading models from recent research trained on much larger datasets. Additionally, with only 8 billion parameters, this relatively compact foundation model can be deployed locally within hospitals, making it a practical solution for environments where APIs and other external services cannot be used due to strict privacy and security requirements.

医学报告生成模型融合小样本学习本地部署

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。