arXiv:2409.00250cs.CV2024-09中稿 · 2024 IEEE Internat…被引 6

把医学报告生成看作多标签分类,提升准确性和效率

Medical Report Generation Is A Multi-label Classification Problem

  • 将报告生成转为多标签分类,利用知识图谱节点增强语义理解
  • 在两个基准数据集上达到当前最优性能,显著超越已有方法
  • 适合医疗AI研究者和临床辅助系统开发者参考

医学报告生成是医疗健康领域的重要任务,旨在从医学影像自动生成详细准确的描述。传统方法将其视为序列生成问题,依赖视觉-语言模型生成连贯报告。本文提出新视角:将该任务重新定义为多标签分类问题,利用常用知识图谱中的放射科节点,通过分类技术更有效捕捉关键信息。为此,我们构建了一种基于BLIP并结合分类关键节点的新框架,实现对医学图像中多个关键特征的精准分类与报告生成。大量实验表明,该方法在两个基准数据集上均达到当前最优(SOTA)性能,显著优于现有方法。结果表明,以创新方法重构传统任务具有巨大潜力,有助于实现更高效、更精确的医学报告生成。

原文摘要 · Abstract (English)

Medical report generation is a critical task in healthcare that involves the automatic creation of detailed and accurate descriptions from medical images. Traditionally, this task has been approached as a sequence generation problem, relying on vision-and-language techniques to generate coherent and contextually relevant reports. However, in this paper, we propose a novel perspective: rethinking medical report generation as a multi-label classification problem. By framing the task this way, we leverage the radiology nodes from the commonly used knowledge graph, which can be better captured through classification techniques. To verify our argument, we introduce a novel report generation framework based on BLIP integrated with classified key nodes, which allows for effective report generation with accurate classification of multiple key aspects within the medical images. This approach not only simplifies the report generation process but also significantly enhances performance metrics. Our extensive experiments demonstrate that leveraging key nodes can achieve state-of-the-art (SOTA) performance, surpassing existing approaches across two benchmark datasets. The results underscore the potential of re-envisioning traditional tasks with innovative methodologies, paving the way for more efficient and accurate medical report generation.

医学报告多标签分类知识图谱BLIP

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。