融合文本与结构数据,用路径签名建模肿瘤患者病程,提升生存预测准确率。
MultiSigBERT: Beyond Survival Analysis through Multimodal and Sequential Modeling in Oncology

- 用路径签名捕捉多模态时序数据的高阶交互关系,无需监督
- 在2500+患者数据上实现C-index 0.743,优于单一模态模型
- 适合关注临床预测与多源医疗数据融合的研究者
机器学习已成为现代医疗的关键工具,整合异构数据可显著提升临床决策。电子健康记录(EHR)包含文本报告、数值指标和结构化变量等互补信息,但多数生存模型仍局限于单模态或忽略时间动态。我们提出MultiSigBERT,一种基于路径签名表示的统一多模态时序生存建模框架。将自由文本报告通过上下文词嵌入提取并平均生成句子嵌入,经各模态专用PCA压缩后与结构化协变量拼接,形成联合时序轨迹,并用来自粗糙路径理论的签名变换编码,无需监督即可高效捕获跨模态的高阶时间交互。最终将签名特征输入LASSO正则化Cox模型,生成个体化风险评分。该方法在里昂贝拉尔中心的真实肿瘤队列中验证,包含超12万份医疗报告和结构化记录,覆盖2500余名患者。独立测试集上达到C-index 0.743(标准差0.029),证明联合建模多模态时序动态与患者层面几何结构对生存预测具有显著优势。
原文摘要 · Abstract (English)
Machine learning has become an essential component of modern healthcare, where the integration of heterogeneous data sources offers unprecedented opportunities to improve clinical decision-making. Electronic Health Records (EHR) contain complementary information -- including narrative clinical reports, numerical measurements, and structured variables -- yet most survival models remain limited to a single modality or fail to exploit the temporal nature of patient trajectories. We propose MultiSigBERT, a unified framework for multimodal sequential survival modeling in oncology based on path signature representations. Here, narrative medical reports (free-text) are converted into sentence embeddings by extracting and averaging contextual word embeddings. These representations are then compressed via modality-specific PCA and concatenated with structured covariates to form joint temporal trajectories which are then encoded using the Signature transform, a tool from Rough Paths theory that efficiently captures higher-order temporal interactions across modalities without supervision needed. The computed Signature features are finally incorporated as high dimensional features into a LASSO-regularized Cox model to estimate individualized risk scores. The performance of our novel MultiSigBERT pipeline is illustrated on the analysis of a real-world oncology cohort from the Léon Bérard Center, comprising over 120,000 medical reports and structured records from more than 2,500 patients. The model achieves a concordance index of 0.743 (sd 0.029) on an independent test set, demonstrating the benefit of jointly modeling multimodal temporal dynamics together with patient-level geometric structure for survival prediction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。