首个系统评估抗癌药物反应预测模型可复用性,推动科研共享标准
Assessing Reusability of Deep Learning-Based Monotherapy Drug Response Prediction Models Trained with Omics Data
- 构建评分体系,从代码、数据、环境三方面评估17个深度学习模型的可重用性
- 发现多数模型存在代码不规范、数据缺失或预处理不透明等问题
- 提出可落地的开源建议,助力精准医疗模型持续优化与社区协作
癌症药物反应预测(DRP)模型为精准肿瘤学提供了前景,有望根据患者个体特征定制治疗方案。尽管深度学习方法在此领域展现出巨大潜力,但真正能应用于临床并揭示治疗响应分子机制的模型,将依赖于协作研究。这凸显了开发可重用、可适配模型的重要性,以供科学界持续改进和验证。本研究提出了一个评估预测模型可重用性的评分系统,并应用于17个经同行评审的深度学习驱动的DRP模型。作为IMPROVE项目的一部分,该项目旨在建立跨科学领域的深度学习模型系统性评估与比较方法,我们聚焦软件环境、代码模块化、数据可用性与预处理三个关键维度进行分析。虽非主要目标,我们也尝试重现关键性能指标以验证模型行为与适应性。对17个模型的评估揭示了其在可重用性上的优势与不足。为促进严谨实践与开源共享,我们提出了模型开发与发布建议。遵循这些建议可有效解决本研究识别出的多数问题,在不显著增加研究人员负担的前提下提升模型可重用性。该工作首次全面评估了多样化DRP模型的可重用性与可再现性,为当前模型共享实践提供洞察,并推动DRP及更广泛的人工智能赋能科学研究社区的标准建设。
原文摘要 · Abstract (English)
Cancer drug response prediction (DRP) models present a promising approach towards precision oncology, tailoring treatments to individual patient profiles. While deep learning (DL) methods have shown great potential in this area, models that can be successfully translated into clinical practice and shed light on the molecular mechanisms underlying treatment response will likely emerge from collaborative research efforts. This highlights the need for reusable and adaptable models that can be improved and tested by the wider scientific community. In this study, we present a scoring system for assessing the reusability of prediction DRP models, and apply it to 17 peer-reviewed DL-based DRP models. As part of the IMPROVE (Innovative Methodologies and New Data for Predictive Oncology Model Evaluation) project, which aims to develop methods for systematic evaluation and comparison DL models across scientific domains, we analyzed these 17 DRP models focusing on three key categories: software environment, code modularity, and data availability and preprocessing. While not the primary focus, we also attempted to reproduce key performance metrics to verify model behavior and adaptability. Our assessment of 17 DRP models reveals both strengths and shortcomings in model reusability. To promote rigorous practices and open-source sharing, we offer recommendations for developing and sharing prediction models. Following these recommendations can address many of the issues identified in this study, improving model reusability without adding significant burdens on researchers. This work offers the first comprehensive assessment of reusability and reproducibility across diverse DRP models, providing insights into current model sharing practices and promoting standards within the DRP and broader AI-enabled scientific research community.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。