用单张胸片生成放射科报告，靠解码器级解剖注意力提升准确性和连贯性。
Radiology Report Generation with Layer-Wise Anatomical Attention
- 在解码器中引入分层解剖注意力，结合肺心分割图增强关键区域关注。
- 在MIMIC-CXR数据集上，5种主要病变的宏平均F1提升168%，整体14项观察提升86%。
- 模型轻量且仅依赖单张图像，适合资源有限的临床环境使用。
自动放射科报告生成是多模态深度学习的有前景应用，旨在减轻报告负担并提高一致性。然而，当前最先进系统（如MAIRA-2和MedPaLM-M）依赖大规模多模态训练、临床元数据及多视角影像，资源消耗大，难以普及。本文提出一种紧凑的图像到文本架构，仅需一张正位胸片即可生成胸片报告的发现部分。模型结合冻结的DINOv3视觉变压器编码器与增强的GPT-2解码器，通过分层高斯平滑整合肺和心脏分割掩码，以无额外可训练参数的方式引导注意力聚焦于临床相关区域。在官方MIMIC-CXR数据集上，采用CheXpert和RadGraph指标评估，模型表现显著提升：5种关键病理的CheXpert宏平均F1从0.083升至0.238（+168%），微平均F1从0.137升至0.337（+146%），14项观测综合得分从0.170升至0.316（+86%）。结构连贯性也明显改善，RadGraph F1提升9.7%。尽管模型小且纯图像条件驱动，结果表明解码器级解剖引导能有效提升空间定位能力与临床相关区域的生成连贯性。源代码已公开于：https://github.com/devMuniz02/UDEM-CXR-Reporting-Thesis-2025。
原文摘要 · Abstract (English)
Automatic radiology report generation is a promising application of multimodal deep learning, aiming to reduce reporting workload and improve consistency. However, current state-of-the-art (SOTA) systems - such as Multimodal AI for Radiology Applications (MAIRA-2) and Medical Pathways Language Model-Multimodal (MedPaLM-M) - depend on large-scale multimodal training, clinical metadata, and multiple imaging views, making them resource-intensive and inaccessible for most settings. We introduce a compact image-to-text architecture that generates the Findings section of chest X-ray reports from a single frontal image. The model combines a frozen Self-Distillation with No Labels v3 (DINOv3) Vision Transformer (ViT) encoder with a Generative Pre-trained Transformer 2 (GPT-2) decoder enhanced by layer-wise anatomical attention. This mechanism integrates lung and heart segmentation masks through hierarchical Gaussian smoothing, biasing attention toward clinically relevant regions without adding trainable parameters. Evaluated on the official Medical Information Mart for Intensive Care-Chest X-ray (MIMIC-CXR) dataset using Chest Radiograph Expert (CheXpert) and Radiology Graph (RadGraph) metrics, our approach achieved substantial gains: CheXpert Macro-F1 for five key pathologies increased by 168% (0.083 -> 0.238) and Micro-F1 by 146% (0.137 -> 0.337), while broader performance across 14 observations improved by 86% (0.170 -> 0.316). Structural coherence also improved, with RadGraph F1 rising by 9.7%. Despite its small size and purely image-conditioned design, the model demonstrates that decoder-level anatomical guidance improves spatial grounding and enhances coherence in clinically relevant regions. The source code is publicly available at: https://github.com/devMuniz02/UDEM-CXR-Reporting-Thesis-2025.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。