用深度学习融合长期体检数据，预测免疫治疗患者存活率。
Multimodal Integration of Longitudinal Noninvasive Diagnostics for Survival Prediction in Immunotherapy Using Deep Learning
- 构建多模态时序注意力网络，整合血液、用药和影像数据。
- 3至12个月存活预测AUC达0.84至0.81，优于传统方法。
- 适合临床个性化预后评估，尤其短期生存预测场景。
免疫疗法革新了癌症治疗格局，但对晚期癌症患者免疫治疗反应模式的理解仍有限。本研究利用人工智能分析常规采集的非侵入性纵向多模态数据，旨在推动个性化治疗。我们提出一种新型神经网络架构——基于多模态变压器的简单时序注意力（MMTSimTA）网络，整合694例接受免疫治疗的泛癌患者治疗前与治疗期的血液指标、用药记录及CT测得的器官体积，用于预测3、6、9和12个月的死亡风险。不同变体的MMTSimTA模型与基于中间融合和晚期融合的基线方法进行比较。结果显示，最优变体在3、6、9、12个月的生存预测中分别取得AUC为0.84±0.04、0.83±0.02、0.82±0.02、0.81±0.03的性能。研究表明，通过新架构整合非侵入性纵向数据可显著提升多模态预后预测能力，尤其在短期生存预测方面表现突出。该研究证明，利用深度学习实现多模态纵向数据融合，为免疫治疗患者的个性化预后评估提供了有前景的新路径。
原文摘要 · Abstract (English)
Purpose: Immunotherapies have revolutionized the landscape of cancer treatments. However, our understanding of response patterns in advanced cancers treated with immunotherapy remains limited. By leveraging routinely collected noninvasive longitudinal and multimodal data with artificial intelligence, we could unlock the potential to transform immunotherapy for cancer patients, paving the way for personalized treatment approaches. Methods: In this study, we developed a novel artificial neural network architecture, multimodal transformer-based simple temporal attention (MMTSimTA) network, building upon a combination of recent successful developments. We integrated pre- and on-treatment blood measurements, prescribed medications and CT-based volumes of organs from a large pan-cancer cohort of 694 patients treated with immunotherapy to predict mortality at three, six, nine and twelve months. Different variants of our extended MMTSimTA network were implemented and compared to baseline methods incorporating intermediate and late fusion based integration methods. Results: The strongest prognostic performance was demonstrated using a variant of the MMTSimTA model with area under the curves (AUCs) of $0.84 \pm $0.04, $0.83 \pm $0.02, $0.82 \pm $0.02, $0.81 \pm $0.03 for 3-, 6-, 9-, and 12-month survival prediction, respectively. Discussion: Our findings show that integrating noninvasive longitudinal data using our novel architecture yields an improved multimodal prognostic performance, especially in short-term survival prediction. Conclusion: Our study demonstrates that multimodal longitudinal integration of noninvasive data using deep learning may offer a promising approach for personalized prognostication in immunotherapy-treated cancer patients.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。