arXiv:2509.12866cs.CV2025-09

用大模型生成狗骨骼疾病可视化数据,解决罕见病数据不足问题。

Leveraging Large Language Models to Effectively Generate Visual Data for Canine Musculoskeletal Diagnoses

  • 将病症标注转为文本描述,通过提示工程生成合成图像数据。
  • 仅用合成数据训练的模型在真实数据上达88%准确率。
  • 适合医疗数据稀缺场景,方法可推广至其他领域。

众所周知,更多数据通常能提升AI模型性能。然而,由于某些情况发生罕见或采集成本高,数据收集存在挑战。本研究针对犬类骨骼肌肉状况的视觉诊断新方法,其中异常以彩色线条标记在狗体图上,对应不同肌肉或关节,可映射至大语言模型(LLM)处理的文本域。我们提出一种将视觉文档分割为200多个标注区域的映射方法,结合引导解码、思维链推理和少样本提示技术,生成1000份髌骨脱位(膝盖错位)的合成视觉文档。分析表明,生成内容对病灶位置和严重程度敏感,且不受犬只性别影响。进一步生成1000份其他诊断的合成文档,构建二分类数据集。仅使用该合成数据训练的模型,在70份真实文档上取得88%的F1分数,证明了大模型生成合成数据在缓解罕见病数据短缺方面的潜力。尽管方法面向医疗领域,但其思路与技术可迁移至其他场景。

原文摘要 · Abstract (English)

It is well-established that more data generally improves AI model performance. However, data collection can be challenging for certain tasks due to the rarity of occurrences or high costs. These challenges are evident in our use case, where we apply AI models to a novel approach for visually documenting the musculoskeletal condition of dogs. Here, abnormalities are marked as colored strokes on a body map of a dog. Since these strokes correspond to distinct muscles or joints, they can be mapped to the textual domain in which large language models (LLMs) operate. LLMs have demonstrated impressive capabilities across a wide range of tasks, including medical applications, offering promising potential for generating synthetic training data. In this work, we investigate whether LLMs can effectively generate synthetic visual training data for canine musculoskeletal diagnoses. For this, we developed a mapping that segments visual documentations into over 200 labeled regions representing muscles or joints. Using techniques like guided decoding, chain-of-thought reasoning, and few-shot prompting, we generated 1,000 synthetic visual documentations for patellar luxation (kneecap dislocation) diagnosis, the diagnosis for which we have the most real-world data. Our analysis shows that the generated documentations are sensitive to location and severity of the diagnosis while remaining independent of the dog's sex. We further generated 1,000 visual documentations for various other diagnoses to create a binary classification dataset. A model trained solely on this synthetic data achieved an F1 score of 88% on 70 real-world documentations. These results demonstrate the potential of LLM-generated synthetic data, which is particularly valuable for addressing data scarcity in rare diseases. While our methodology is tailored to the medical domain, the insights and techniques can be adapted to other fields.

大模型合成数据医疗影像犬类诊断

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。