用临床文本预测癌症分期,模型表现有潜力但泛化能力待提升。
CaresAI at SMM4H-HeaRD 2026: Predicting TNM Staging
- 分三任务独立预测TNM分期,结合多种预训练模型与经典算法
- LightGBM+TF-IDF在训练集上达AUROC 0.95(N期),测试集宏F1最高0.978
- 模型对长文本和数据不均衡敏感,真实场景应用仍需优化
本研究针对SMM4H-HeaRD 2026第六项共享任务——基于癌症基因组图谱(TCGA)病理报告预测肿瘤、淋巴结和转移(TNM)分期。将问题建模为三个独立的多标签分类任务,比较了传统机器学习与深度学习方法,采用TF-IDF特征及ClinicalBERT、BioBERT、PubMedBERT等嵌入表示,配合逻辑回归(LR)、LightGBM、前馈神经网络(FFNN)和宽残差网络(WRN)。实验显示,单个嵌入表示性能接近,组合后提升预测效果。其中,WRN在训练阶段获得T、N、M期的AUROC分别为0.839、0.8502、0.803,对应F1为0.622、0.702、0.9337;而LightGBM+TF-IDF在训练阶段表现最优,各期AUROC为0.9368、0.9524、0.8311,F1为0.7559、0.7384、0.7017。在Codabench测试集上,测试集1的宏F1为0.978(T)、0.957(N)、0.879(M);测试集2为0.807(T)、0.767(N)、1.0(M)。然而,从测试集1到2整体宏F1由0.938降至0.858,表明模型泛化能力有限,对类别不平衡和长文档处理存在挑战。尽管提供了可复现的基线框架,但仍需进一步优化才能应用于真实临床环境。
原文摘要 · Abstract (English)
This study aims to predict Tumor, Node, and Metastasis (TNM) stage labels independently, with the Cancer Genome Atlas (TCGA) pathology report as the sixth shared task of SMM4H-HeaRD 2026. The problem is framed as three multi-label classification tasks. We explore both classical and deep learning approaches using Term Frequency-Inverse Document Frequency (TF-IDF) features and embeddings from ClinicalBERT, BioBERT, and PubMedBERT. These representations are used with Logistic Regression (LR), Light Gradient Boosting Machine (LightGBM), Feed-Forward Neural Networks (FFNN), and Wide Residual Networks (WRN). Our results show that individual embeddings perform similarly to the TNM label classification, while their combination improves its predictive ability. WRN achieves AUROC scores of 0.839 (T), 0.8502 (N), and 0.803 (M) with F1-scores of 0.622, 0.702, and 0.9337, respectively, for the training phase. LightGBM with TF-IDF performs best with AUROC scores of 0.9368 (T), 0.9524 (N), and 0.8311 (M) and F1-scores of 0.7559 (T), 0.7384 (N), and 0.7017 (M) during the training phase. Furthermore, the result of the Codabench for the test sets indicates a Macro-F1 score of 0.978, 0.957, and 0.879 for the T, N, and M categories respectively for test set 1; while test set 2 records a Macro-F1 score for T, N, and M is 0.807, 0.767, 1.0 respectively. However, performance declined during the evaluation phase of the test sets, a drop from 0.938 to 0.858 of test set 1 to 2, for the Macro-F1 score across all stages; suggesting limitations in model generalizability, sensitivity to class imbalance, and challenges in processing lengthy clinical documents. Although this study provides an efficient baseline model and a reproducible pipeline, further optimization and validation are required before it can be considered suitable for use in a real-world clinical setting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。