统一模型实现血液病理多任务分析,提升诊断精度与可解释性
Uni-Hema: Unified Model for Digital Hematopathology
- 构建跨任务、跨模态的统一框架,融合检测、分类、分割等能力
- 在70万+图像和2.1万问答对上训练,单细胞级分析性能优于单一任务模型
- 适用于白血病、疟疾等多种血液病,适合医学影像智能辅助诊断研究
数字血液病理学需在多种疾病类别中进行细胞级分析,包括恶性疾病(如白血病)、感染性疾病(如疟疾)及非恶性红细胞疾病(如镰状细胞病)。现有单任务、视觉语言、全切片图像优化或单细胞血液学模型均存在局限:无法在复杂血液病理中实现统一的多任务、多模态推理。为此,我们提出Uni-Hema,一个集成检测、分类、分割、形态预测与跨疾病推理的多任务统一模型。Uni-Hema基于Hema-Former模块,利用46个公开数据集(超70万张图像,2.1万个问答对),在不同粒度下实现视觉与文本表示的层级融合。大量实验表明,Uni-Hema在各类血液病理任务中表现媲美甚至超越单任务、单数据集模型,同时提供可解释的、与形态相关的单细胞级洞察。该框架为多任务、多模态数字血液病理学建立了新标准。代码将公开。
原文摘要 · Abstract (English)
Digital hematopathology requires cell-level analysis across diverse disease categories, including malignant disorders (e.g., leukemia), infectious conditions (e.g., malaria), and non-malignant red blood cell disorders (e.g., sickle cell disease). Whether single-task, vision-language, WSI-optimized, or single-cell hematology models, these approaches share a key limitation, they cannot provide unified, multi-task, multi-modal reasoning across the complexities of digital hematopathology. To overcome these limitations, we propose Uni-Hema, a multi-task, unified model for digital hematopathology integrating detection, classification, segmentation, morphology prediction, and reasoning across multiple diseases. Uni-Hema leverages 46 publicly available datasets, encompassing over 700K images and 21K question-answer pairs, and is built upon Hema-Former, a multimodal module that bridges visual and textual representations at the hierarchy level for the different tasks (detection, classification, segmentation, morphology, mask language modeling and visual question answer) at different granularity. Extensive experiments demonstrate that Uni-Hema achieves comparable or superior performance to train on a single-task and single dataset models, across diverse hematological tasks, while providing interpretable, morphologically relevant insights at the single-cell level. Our framework establishes a new standard for multi-task and multi-modal digital hematopathology. The code will be made publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。