提出新基准框架DDI-Ben,评估药物相互作用预测在真实分布变化下的表现。
Benchmarking drug-drug interaction prediction methods: a perspective of distribution changes
- 构建模拟分布变化的框架,更贴近真实药物研发场景。
- 多数现有方法在分布变化下性能大幅下降,尤其依赖静态数据。
- 大语言模型与文本信息融合方法表现出更强鲁棒性,适合未来研究。
背景:新兴药物相互作用(DDI)预测对新药研发至关重要,但现实场景中已知药物与新药之间存在分布差异,而当前评估常忽略此问题,因缺乏药物审批数据导致采用不切实际的独立同分布(i.i.d.)划分。结果:我们提出DDI-Ben,一个面向分布变化下新兴DDI预测的基准框架。该框架引入分布变化模拟机制,以药物集合间的分布差异作为真实世界分布变化的代理,并兼容多种药物划分策略。通过对十种代表性方法的广泛评测,发现大多数现有方法在分布变化下性能显著下降。分析表明,基于大语言模型(LLM)的方法及整合药物相关文本信息的方案展现出更好的鲁棒性。为支持后续研究,我们公开了带有模拟分布变化的基准数据集。总体而言,DDI-Ben强调了显式处理分布变化的重要性,并为开发更具韧性的一体化预测方法奠定了基础。代码与数据已开源:https://github.com/LARS-research/DDI-Bench。
原文摘要 · Abstract (English)
Motivation: Emerging drug-drug interaction (DDI) prediction is crucial for new drugs but is hindered by distribution changes between known and new drugs in real-world scenarios. Current evaluation often neglects these changes, relying on unrealistic i.i.d. split due to the absence of drug approval data. Results: We propose DDI-Ben, a benchmarking framework for emerging DDI prediction under distribution changes. DDI-Ben introduces a distribution change simulation framework that leverages distribution changes between drug sets as a surrogate for real-world distribution changes of DDIs, and is compatible with various drug split strategies. Through extensive benchmarking on ten representative methods, we show that most existing approaches suffer substantial performance degradation under distribution changes. Our analysis further indicates that large language model (LLM) based methods and the integration of drug-related textual information offer promising robustness against such degradation. To support future research, we release the benchmark datasets with simulated distribution changes. Overall, DDI-Ben highlights the importance of explicitly addressing distribution changes and provides a foundation for developing more resilient methods for emerging DDI prediction. Availability and implementation: Our code and data are available at https://github.com/LARS-research/DDI-Bench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。