用开源大模型构建多智能体系统，提升基因组问答的效率与可扩展性。
Beyond GeneGPT: A Multi-Agent Architecture with Open-Source LLMs for Enhanced Genomic Question Answering
- 设计模块化多智能体架构，分工完成工具调用、问题生成与结果验证。
- 在90%以上任务中表现优于或持平GeneGPT，平均得分达0.849（Gene-Turing）。
- 使用小型开源模型无需微调，延迟降低40%-50%，适合科研与医疗场景。
基因组问答常需跨多种生物医学数据源进行复杂推理。GeneGPT通过结合领域专用API与OpenAI的code-davinci-002大模型，实现对基因组数据库的自然语言交互。但其依赖专有模型，限制了可扩展性，推高运营成本，并引发数据隐私与泛化能力担忧。本文以Llama 3.1、Qwen2.5及Qwen2.5 Coder等开源模型，在单体架构下复现并分析GeneGPT，揭示其局限性。在此基础上，我们提出OpenBioLLM——一个模块化多智能体框架，引入任务专业化机制，实现工具路由、查询生成与响应验证的协同推理与角色化执行。OpenBioLLM在超过90%的基准任务上达到或超越GeneGPT性能，基因图灵测试平均得分为0.849，GeneHop测试得分为0.830，且仅使用更小的开源模型，无额外微调或工具预训练。其多智能体设计使各任务延迟降低40%-50%，显著提升效率，同时保持模型能力。全面评估表明，开源多智能体系统在基因组问答中具有巨大潜力。代码与资源已公开于https://github.com/ielab/OpenBioLLM。
原文摘要 · Abstract (English)
Genomic question answering often requires complex reasoning and integration across diverse biomedical sources. GeneGPT addressed this challenge by combining domain-specific APIs with OpenAI's code-davinci-002 large language model to enable natural language interaction with genomic databases. However, its reliance on a proprietary model limits scalability, increases operational costs, and raises concerns about data privacy and generalization. In this work, we revisit and reproduce GeneGPT in a pilot study using open source models, including Llama 3.1, Qwen2.5, and Qwen2.5 Coder, within a monolithic architecture; this allows us to identify the limitations of this approach. Building on this foundation, we then develop OpenBioLLM, a modular multi-agent framework that extends GeneGPT by introducing agent specialization for tool routing, query generation, and response validation. This enables coordinated reasoning and role-based task execution. OpenBioLLM matches or outperforms GeneGPT on over 90% of the benchmark tasks, achieving average scores of 0.849 on Gene-Turing and 0.830 on GeneHop, while using smaller open-source models without additional fine-tuning or tool-specific pretraining. OpenBioLLM's modular multi-agent design reduces latency by 40-50% across benchmark tasks, significantly improving efficiency without compromising model capability. The results of our comprehensive evaluation highlight the potential of open-source multi-agent systems for genomic question answering. Code and resources are available at https://github.com/ielab/OpenBioLLM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。