开源大模型经领域微调后，在抗疟药物虚拟筛选中表现超越经典与商业模型。
Rigorous Evaluation of Large Language Models for Malaria Drug Discovery: Trade-offs in Performance, Scale, and Resource Utility
- 用疟疾药物数据集微调大模型，提升虚拟筛选准确性。
- 微调后模型最高达ROC-AUC 0.731，EF@1%达4.99，显著优于基线。
- 化学领域预训练对前瞻性发现至关重要，适合资源有限的研究团队。
我们构建了Malaria-Instruct数据集，基于ChEMBL Legacy Malaria数据集用于疟疾虚拟筛选，并系统评估了五个开源LLM（Gemma-2 2B/9B、TxGemma-2B/9B、LlaSMol-Mistral-7B）在严格的分布外数据划分下的表现。对比了随机森林、XGBoost等经典模型以及Gemini 2.5、OpenAI o3等前沿闭源模型，在少样本条件下，微调后的模型显著优于所有基线：TxGemma-9B达到最高ROC-AUC（0.731 ± 0.005），LlaSMol-Mistral-7B实现最佳富集因子（EF@1% ≈ 4.99）。无领域微调时，即使顶级闭源模型也仅获约0.53–0.59的ROC-AUC，无法可靠区分。生物医学预训练带来可观优势，而化学感知预训练进一步提升前瞻富集能力。微调后的开源模型在资源效率与性能间取得平衡，是抗疟虚拟筛选的理想方案。
原文摘要 · Abstract (English)
We introduce Malaria-Instruct, a curated instruction-following dataset derived from the ChEMBL Legacy Malaria corpus for Malaria virtual screening, and conduct a systematic evaluation of five open-source LLMs; Gemma-2 2B/9B, TxGemma-2B/9B, and LlaSMol-Mistral-7B, on a rigorous out-of-distribution data split. Performance was benchmarked against classical ML models (Random Forest, XGBoost) and frontier proprietary models (Gemini 2.5, OpenAI o3) under few-shot conditions. Fine-tuned LLMs substantially outperformed all baselines: TxGemma-9B achieved the highest ROC-AUC ($0.731 \pm 0.005$) and LlaSMol-Mistral-7B the best enrichment factor (EF@1\% $\approx$ 4.99). Domain-specific fine-tuning proved categorically indispensable with TxGemma-9B collapsing from ROC-AUC 0.731 to 0.499, under its best few-shot condition, and neither Gemini 2.5 (ROC-AUC $\approx$ 0.53) nor o3 (ROC-AUC $\approx$ 0.59) achieved reliable discrimination without fine-tuning. Biomedical pretraining conferred a measurable advantage at equivalent scale, while chemistry-aware pretraining yielded superior prospective enrichment. Fine-tuned open-source LLMs represent a compelling, resource-efficient paradigm for antimalarial VS, outperforming both classical pipelines and proprietary reasoning models under structurally challenging conditions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。