用多模态大模型自动解析化学反应图像,提升数据提取效率。
Towards Large-scale Chemical Reaction Image Parsing via a Multimodal Large Language Model
- 构建首个专用于化学反应图像解析的多模态大模型RxnIM。
- 在多个基准上平均F1达88%,比现有方法高5%。
- 适合化学信息学、AI制药等领域研究人员使用。
人工智能在有机化学研究中展现出巨大潜力,但其效果依赖于高质量化学反应数据的可用性。目前,大多数已发表的化学反应未以机器可读形式呈现,限制了AI在该领域的广泛应用。化学反应数据的结构化提取仍高度依赖人工整理,而从化学反应图像中自动解析出机器可读数据仍是重大挑战。为此,我们提出反应图像多模态大语言模型(RxnIM),这是首个专门设计用于将化学反应图像转化为机器可读反应数据的多模态大模型。RxnIM不仅能提取图像中的关键化学成分,还能理解描述反应条件的文本内容。结合专门设计的大规模数据集生成方法支持模型训练,我们的方法在多个基准测试中平均F1得分达到88%,优于已有文献方法5个百分点。这标志着向从化学文献图像中自动构建大规模可机读反应数据库迈出了关键一步,为化学领域AI研究提供了重要数据资源。本工作发布的源代码、模型检查点和数据集均采用宽松许可协议开源。RxnIM网页应用实例可通过https://huggingface.co/spaces/CYF200127/RxnIM访问。
原文摘要 · Abstract (English)
Artificial intelligence (AI) has demonstrated significant promise in advancing organic chemistry research; however, its effectiveness depends on the availability of high-quality chemical reaction data. Currently, most published chemical reactions are not available in machine-readable form, limiting the broader application of AI in this field. The extraction of published chemical reactions into structured databases still relies heavily on manual curation, and robust automatic parsing of chemical reaction images into machine-readable data remains a significant challenge. To address this, we introduce the Reaction Image Multimodal large language model (RxnIM), the first multimodal large language model specifically designed to parse chemical reaction images into machine-readable reaction data. RxnIM not only extracts key chemical components from reaction images but also interprets the textual content that describes reaction conditions. Together with specially designed large-scale dataset generation method to support model training, our approach achieves excellent performance, with an average F1 score of 88% on various benchmarks, surpassing literature methods by 5%. This represents a crucial step toward the automatic construction of large databases of machine-readable reaction data parsed from images in the chemistry literature, providing essential data resources for AI research in chemistry. The source code, model checkpoints, and datasets developed in this work are released under permissive licenses. An instance of the RxnIM web application can be accessed at https://huggingface.co/spaces/CYF200127/RxnIM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。