基于云的多智能体系统,让AI自动完成科研中的复杂任务。
A Cloud-based Multi-Agentic Workflow for Science
- 用监督智能体协调多个专用智能体,实现任务自动分配与执行。
- 在合成数据上任务完成率达97.5%,真实科研任务完成率达91%。
- 适合化学、材料等领域的研究人员快速部署自动化科研助手。
随着大语言模型(LLMs)在各科学领域广泛应用,其无法执行复杂任务(如模拟运行或复杂决策)的局限性限制了实际应用。基于LLM的智能体通过调用外部资源和工具,可弥补这一缺陷,正迅速普及。然而,如何在模型、云服务商和外部资源间平衡,设计出高效的工作流,仍是实施智能体系统的重大挑战。本文提出一种领域无关、模型无关的云原生多智能体工作流框架,作为科学助手全程运行于云端。该框架由监督智能体协调具备不同能力的多个智能体,将文献综述、数据分析等简单任务与模拟运行等复杂任务统一整合。我们构建了一个概念验证系统,用于加速催化剂研究——该课题在化学与材料科学中至关重要。报告了系统的运行成本及各服务使用成本构成。在自定义合成基准和主流化学基准上评估系统表现,并进行专家验证。结果显示,系统能以90%的准确率正确路由任务,在合成任务中成功完成率为97.5%,真实任务中为91%,且准确率优于或媲美多数前沿模型,证明该框架在其他科学领域具有高度可复制性。
原文摘要 · Abstract (English)
As Large Language Models (LLMs) become ubiquitous across various scientific domains, their lack of ability to perform complex tasks like running simulations or to make complex decisions limits their utility. LLM-based agents bridge this gap due to their ability to call external resources and tools and thus are now rapidly gaining popularity. However, coming up with a workflow that can balance the models, cloud providers, and external resources is very challenging, making implementing an agentic system more of a hindrance than a help. In this work, we present a domain-agnostic, model-independent workflow for an agentic framework that can act as a scientific assistant while being run entirely on cloud. Built with a supervisor agent marshaling an array of agents with individual capabilities, our framework brings together straightforward tasks like literature review and data analysis with more complex ones like simulation runs. We describe the framework here in full, including a proof-of-concept system we built to accelerate the study of Catalysts, which is highly important in the field of Chemistry and Material Science. We report the cost to operate and use this framework, including the breakdown of the cost by services use. We also evaluate our system on a custom-curated synthetic benchmark and a popular Chemistry benchmark, and also perform expert validation of the system. The results show that our system is able to route the task to the correct agent 90% of the time and successfully complete the assigned task 97.5% of the time for the synthetic tasks and 91% of the time for real-world tasks, while still achieving better or comparable accuracy to most frontier models, showing that this is a viable framework for other scientific domains to replicate.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。