融合分子与细胞视频信息，提升药物作用机制识别准确率。
MolCLIP: A Molecular-Auxiliary CLIP Framework for Identifying Drug Mechanism of Action Based on Time-Lapsed Mitochondrial Images
- 用分子信息辅助视频特征学习，构建多模态视觉语言模型。
- 在MitoDataset上，药物识别mAP提升51.2%，机制识别提升20.5%。
- 适合药物机理研究、AI制药及活细胞动态分析领域的研究人员。
药物作用机制（MoA）研究药物分子如何与细胞相互作用，对药物研发和临床应用至关重要。近年来，深度学习模型通过高内涵荧光细胞图像识别MoA，但多关注空间特征，忽视活细胞的时序动态。时间延时成像更利于观察药物反应，且药物可引发特定机制相关的细胞动态变化，表明分子模态可补充图像信息。本文提出首个结合显微细胞视频与分子模态的MolCLIP框架，设计分子辅助的CLIP架构，引导视频特征学习分子潜在空间分布，并引入度量学习优化视频特征聚合。在MitoDataset上的实验表明，MolCLIP在药物识别和MoA识别任务中分别实现51.2%和20.5%的mAP提升。
原文摘要 · Abstract (English)
Drug Mechanism of Action (MoA) mainly investigates how drug molecules interact with cells, which is crucial for drug discovery and clinical application. Recently, deep learning models have been used to recognize MoA by relying on high-content and fluorescence images of cells exposed to various drugs. However, these methods focus on spatial characteristics while overlooking the temporal dynamics of live cells. Time-lapse imaging is more suitable for observing the cell response to drugs. Additionally, drug molecules can trigger cellular dynamic variations related to specific MoA. This indicates that the drug molecule modality may complement the image counterpart. This paper proposes MolCLIP, the first visual language model to combine microscopic cell video- and molecule-modalities. MolCLIP designs a molecule-auxiliary CLIP framework to guide video features in learning the distribution of the molecular latent space. Furthermore, we integrate a metric learning strategy with MolCLIP to optimize the aggregation of video features. Experimental results on the MitoDataset demonstrate that MolCLIP achieves improvements of 51.2% and 20.5% in mAP for drug identification and MoA recognition, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。