用视觉Transformer统一分析病理切片，实现乳腺癌HER2蛋白表达的精准像素级评分
Feature Learning with Multi-Stage Vision Transformers on Inter-Modality HER2 Status Scoring and Tumor Classification on Whole Slides
- 基于多阶段ViT架构，联合处理H&E与IHC切片，定位肿瘤并映射对应区域
- 4类HER2评分准确率达94%，特异性达93.3%，可区分阴性与阳性
- 端到端输出像素级标注，结果接近人类病理医生判断水平
组织病理图像（如H&E染色）在肿瘤检测中已被广泛应用。然而，指导治疗需精确评估人表皮生长因子受体2（HER2）蛋白表达水平。预测低或高表达均具挑战性，且联合分析H&E与免疫组化（IHC）图像进行HER2评分困难。尽管已有深度学习方法尝试解决该问题，但普遍缺乏像素级定位能力。本文提出一种端到端的视觉变压器系统，用于全切片图像（WSIs）上的HER2评分与肿瘤分类。方法包含对H&E WSI的分块处理以定位肿瘤；设计新型映射函数，将恶性区域与对应IHC区域匹配；嵌入临床启发式评分机制，实现四分类HER2评分（0, 1+, 2+, 3+），并支持像素级标注。研究使用13例患者共有的私有化H&E与IHC WSI数据集，实验表明：肿瘤定位效果良好；HER2状态预测准确率为0.94，特异性为0.933；通过切片块对比验证，该方法在端到端ViT模型上有效整合双模态图像进行HER2评分。
原文摘要 · Abstract (English)
The popular use of histopathology images, such as hematoxylin and eosin (H&E), has proven to be useful in detecting tumors. However, moving such cancer cases forward for treatment requires accurate on the amount of the human epidermal growth factor receptor 2 (HER2) protein expression. Predicting both the lower and higher levels of HER2 can be challenging. Moreover, jointly analyzing H&E and immunohistochemistry (IHC) stained images for HER2 scoring is difficult. Although several deep learning methods have been investigated to address the challenge of HER2 scoring, they suffer from providing a pixel-level localization of HER2 status. In this study, we propose a single end-to-end pipeline using a system of vision transformers with HER2 status scoring on whole slide images of WSIs. The method includes patch-wise processing of H&E WSIs for tumor localization. A novel mapping function is proposed to correspondingly identify correlated IHC WSIs regions with malignant regions on H&E. A clinically inspired HER2 scoring mechanism is embedded in the pipeline and allows for automatic pixel-level annotation of 4-way HER2 scoring (0, 1+, 2+, and 3+). Also, the proposed method accurately returns HER2-negative and HER2-positive. Privately curated datasets were collaboratively extracted from 13 different cases of WSIs of H&E and IHC. A thorough experiment is conducted on the proposed method. Results obtained showed a good classification accuracy during tumor localization. Also, a classification accuracy of 0.94 and a specificity of 0.933 were returned for the prediction of HER2 status, scoring in the 4-way methods. The applicability of the proposed pipeline was investigated using WSIs patches as comparable to human pathologists. Findings from the study showed the usability of jointly evaluated H&E and IHC images on end-to-end ViTs-based models for HER2 scoring
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。