用稀疏注意力提升药物毒性预测的准确率与可解释性
Task-Specific Sparse Feature Masks for Molecular Toxicity Prediction with Chemical Language Models
- 为每种毒性任务设计稀疏注意力模块，聚焦关键分子片段
- 在三个数据集上均优于单任务与传统多任务模型
- 生成化学直观的可视化结果，适合需要透明决策的药企
可靠的体外分子毒性预测是现代药物发现的核心，为实验筛选提供了可扩展的替代方案。然而，当前顶尖模型的黑箱特性仍是实际应用的主要障碍，因为高风险安全决策需要可验证的结构洞察。为此，我们提出一种新型多任务学习框架，旨在同时提升预测准确率与可解释性。该架构融合共享的化学语言模型与任务特定注意力模块，并对这些模块施加L1稀疏惩罚，使其仅关注每个毒性终点的少数关键分子片段。框架端到端训练，可适配多种基于Transformer的骨干网络。在ClinTox、SIDER和Tox21基准数据集上的评估显示，该方法持续优于单任务及标准多任务基线。关键的是，稀疏注意力权重提供了化学上直观的可视化，揭示了影响预测的具体分子片段，从而增强对模型决策过程的理解。
原文摘要 · Abstract (English)
Reliable in silico molecular toxicity prediction is a cornerstone of modern drug discovery, offering a scalable alternative to experimental screening. However, the black-box nature of state-of-the-art models remains a significant barrier to adoption, as high-stakes safety decisions demand verifiable structural insights alongside predictive performance. To address this, we propose a novel multi-task learning (MTL) framework designed to jointly enhance accuracy and interpretability. Our architecture integrates a shared chemical language model with task-specific attention modules. By imposing an L1 sparsity penalty on these modules, the framework is constrained to focus on a minimal set of salient molecular fragments for each distinct toxicity endpoint. The resulting framework is trained end-to-end and is readily adaptable to various transformer-based backbones. Evaluated on the ClinTox, SIDER, and Tox21 benchmark datasets, our approach consistently outperforms both single-task and standard MTL baselines. Crucially, the sparse attention weights provide chemically intuitive visualizations that reveal the specific fragments influencing predictions, thereby enhancing insight into the model's decision-making process.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。