用可自更新的化学知识库提升大模型推理能力，解决错误累积问题
ChemAgent: Self-updating Library in Large Language Models Improves Chemical Reasoning
- 将化学任务拆解为子任务，构建可检索的动态知识库
- 在4个数据集上使GPT-4性能提升最高达46%
- 适合需要精准推理的药物发现与材料科学场景
化学推理通常涉及复杂的多步过程，需精确计算，微小错误即可引发连锁失效。大型语言模型（LLMs）在处理领域特定公式、准确执行推理步骤以及有效整合代码方面存在困难。为此，我们提出ChemAgent，一种通过动态自更新知识库提升LLM性能的新框架。该知识库通过将化学任务分解为子任务并整理成结构化集合，供未来查询参考。面对新问题时，ChemAgent从知识库中检索并精炼相关信息（称为记忆），促进有效任务分解与解决方案生成。方法设计了三种记忆类型和一个增强型推理组件，使LLM能通过经验持续改进。在SciBench中的四个化学推理数据集上，ChemAgent表现显著优于现有方法，最高实现46%的性能提升（GPT-4）。研究结果表明其在药物发现与材料科学等任务中具有广泛应用潜力。
原文摘要 · Abstract (English)
Chemical reasoning usually involves complex, multi-step processes that demand precise calculations, where even minor errors can lead to cascading failures. Furthermore, large language models (LLMs) encounter difficulties handling domain-specific formulas, executing reasoning steps accurately, and integrating code effectively when tackling chemical reasoning tasks. To address these challenges, we present ChemAgent, a novel framework designed to improve the performance of LLMs through a dynamic, self-updating library. This library is developed by decomposing chemical tasks into sub-tasks and compiling these sub-tasks into a structured collection that can be referenced for future queries. Then, when presented with a new problem, ChemAgent retrieves and refines pertinent information from the library, which we call memory, facilitating effective task decomposition and the generation of solutions. Our method designs three types of memory and a library-enhanced reasoning component, enabling LLMs to improve over time through experience. Experimental results on four chemical reasoning datasets from SciBench demonstrate that ChemAgent achieves performance gains of up to 46% (GPT-4), significantly outperforming existing methods. Our findings suggest substantial potential for future applications, including tasks such as drug discovery and materials science. Our code can be found at https://github.com/gersteinlab/chemagent
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。