让AI自动把论文代码变成可调用工具，提升科研自动化水平
LLM Agents Making Agent Tools
- 通过闭环自修正机制，自动安装依赖并生成可用代码
- 在15个任务上正确实现80%的复杂计算任务，优于现有方法
- 适合科研人员快速构建专用工具，尤其医学与生命科学领域
工具使用使大型语言模型（LLMs）能够通过动态调用外部软件组件执行复杂多步任务。然而，这些工具需由人类开发者提前实现，限制了其在需大量专业工具的领域（如生命科学和医学）的应用。受越来越多科学论文附带公开代码库的趋势启发，我们提出 ToolMaker，一个能自主将含代码的论文转化为 LLM 可用工具的智能体框架。给定 GitHub 链接和简短任务描述，ToolMaker 自动安装依赖并生成完成任务的代码，采用闭环自修正机制进行调试。为评估该方法，我们构建了一个涵盖多个领域的15个复杂计算任务基准，包含100多个单元测试，以检验正确性和鲁棒性。实验表明，该方法成功实现80%的任务，显著优于当前最先进的软件工程智能体。ToolMaker 朝着完全自主的基于智能体的科学工作流迈进一步。代码与基准已开源：https://github.com/KatherLab/ToolMaker。
原文摘要 · Abstract (English)
Tool use has turned large language models (LLMs) into powerful agents that can perform complex multi-step tasks by dynamically utilising external software components. However, these tools must be implemented in advance by human developers, hindering the applicability of LLM agents in domains demanding large numbers of highly specialised tools, like in life sciences and medicine. Motivated by the growing trend of scientific studies accompanied by public code repositories, we propose ToolMaker, an agentic framework that autonomously transforms papers with code into LLM-compatible tools. Given a GitHub URL and short task description, ToolMaker autonomously installs dependencies and generates code to perform the task, using a closed-loop self-correction mechanism for debugging. To evaluate our approach, we introduce a benchmark comprising 15 complex computational tasks spanning various domains with over 100 unit tests to assess correctness and robustness. Our method correctly implements 80% of the tasks, substantially outperforming current state-of-the-art software engineering agents. ToolMaker therefore is a step towards fully autonomous agent-based scientific workflows. Our code and benchmark are publicly available at https://github.com/KatherLab/ToolMaker.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。