用Transformer统一处理3D医学影像的检测、分割和分类,提升效率。
MTMed3D: A Multi-Task Transformer-Based Model for 3D Medical Imaging
- 采用共享Transformer编码器+任务专用解码器的多任务架构。
- 在BraTS 2018/2019上三项任务均表现优异,检测性能优于已有方法。
- 多任务联合训练大幅降低计算开销,推理更快,适合临床部署。
在医学影像领域,人工智能辅助的物体检测、分割和分类技术被广泛应用以减轻医生负担。然而,当前多数模型为单任务设计,忽视了任务间的共享信息,导致实际应用中效率低下。本文提出MTMed3D,一种基于Transformer的端到端多任务模型,可同时完成3D医学影像中的检测、分割与分类。模型采用Transformer作为共享编码器提取多尺度特征,再通过CNN-based任务专用解码器完成具体任务。在BraTS 2018和2019数据集上的实验表明,该模型在三项任务上均取得良好结果,尤其在检测任务中优于先前工作。与独立训练的单任务模型相比,多任务模型显著降低计算成本,实现更快推理速度,且性能相当,凸显其高效优势。据我们所知,这是首个将Transformer用于3D医学影像中同时涵盖检测、分割与分类的多任务学习工作,具有提升诊疗流程的潜力。代码已公开于https://github.com/fanlimua/MTMed3D.git。
原文摘要 · Abstract (English)
In the field of medical imaging, AI-assisted techniques such as object detection, segmentation, and classification are widely employed to alleviate the workload of physicians and doctors. However, single-task models are predominantly used, overlooking the shared information across tasks. This oversight leads to inefficiencies in real-life applications. In this work, we propose MTMed3D, a novel end-to-end Multi-task Transformer-based model to address the limitations of single-task models by jointly performing 3D detection, segmentation, and classification in medical imaging. Our model uses a Transformer as the shared encoder to generate multi-scale features, followed by CNN-based task-specific decoders. The proposed framework was evaluated on the BraTS 2018 and 2019 datasets, achieving promising results across all three tasks, especially in detection, where our method achieves better results than prior works. Additionally, we compare our multi-task model with equivalent single-task variants trained separately. Our multi-task model significantly reduces computational costs and achieves faster inference speed while maintaining comparable performance to the single-task models, highlighting its efficiency advantage. To the best of our knowledge, this is the first work to leverage Transformers for multi-task learning that simultaneously covers detection, segmentation, and classification tasks in 3D medical imaging, presenting its potential to enhance diagnostic processes. The code is available at https://github.com/fanlimua/MTMed3D.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。