LLM可设计出高毒性生物制剂,暴露当前技术的双重用途风险。
Can Large Language Models Design Biological Weapons? Evaluating Moremi Bio
- 用无安全限制的LLM生成毒蛋白与小分子,模拟生物武器设计。
- 1020种毒蛋白均显示高毒性,部分接近蓖麻毒素等已知毒素。
- 警示非专业人员也可滥用该技术,需强化生物安全治理。
人工智能,特别是大语言模型(LLMs),已将药物研发周期缩短高达40%,并提升分子靶点识别能力。然而,这些进展也引发双重用途担忧,因可被用于设计有毒化合物。在未设安全防护的情况下,通过提示Moremi Bio Agent专门设计新型有毒物质,本研究生成了1020种新型毒蛋白和5000种有毒小分子。深入的计算毒性评估表明,所有蛋白质均显示高毒性,其中若干接近已知毒素,如蓖麻毒素、白喉毒素及基于解整合素的蛇毒蛋白。部分新化合物与多种已知毒素相似,包括解整合素型埃里索坦、金属蛋白酶、三氟拉文、蛇毒金属蛋白酶及棒状杆菌溃疡毒素。通过定量风险评估与情景分析,我们识别出现有LLM驱动的生物设计流程中的双重用途潜力,并提出多层次缓解策略。该毒性评估结果挑战了‘大语言模型无法设计生物武器’的论断,凸显其在生物设计中潜在的滥用风险,对研发造成重大威胁。该技术对缺乏技术背景者亦具可及性,带来严重生物安全风险。研究强调亟需建立强有力的治理机制与技术防护措施,以平衡快速生物技术创新与生物安全需求。
原文摘要 · Abstract (English)
Advances in AI, particularly LLMs, have dramatically shortened drug discovery cycles by up to 40% and improved molecular target identification. However, these innovations also raise dual-use concerns by enabling the design of toxic compounds. Prompting Moremi Bio Agent without the safety guardrails to specifically design novel toxic substances, our study generated 1020 novel toxic proteins and 5,000 toxic small molecules. In-depth computational toxicity assessments revealed that all the proteins scored high in toxicity, with several closely matching known toxins such as ricin, diphtheria toxin, and disintegrin-based snake venom proteins. Some of these novel agents showed similarities with other several known toxic agents including disintegrin eristostatin, metalloproteinase, disintegrin triflavin, snake venom metalloproteinase, corynebacterium ulcerans toxin. Through quantitative risk assessments and scenario analyses, we identify dual-use capabilities in current LLM-enabled biodesign pipelines and propose multi-layered mitigation strategies. The findings from this toxicity assessment challenge claims that large language models (LLMs) are incapable of designing bioweapons. This reinforces concerns about the potential misuse of LLMs in biodesign, posing a significant threat to research and development (R&D). The accessibility of such technology to individuals with limited technical expertise raises serious biosecurity risks. Our findings underscore the critical need for robust governance and technical safeguards to balance rapid biotechnological innovation with biosecurity imperatives.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。