研究生成式AI在自主机器中的安全风险，提出安全评分卡方案。
Generative AI Agents in Autonomous Machines: A Safety Perspective
- 将生成式AI作为代理集成到物理自主系统，评估其安全挑战。
- 强调高风险场景下需建立全面的安全保障机制。
- 适合关注智能机器人安全与可信部署的研究者和工程师。
将生成式人工智能（Generative AI）融入自主机器标志着系统运作方式的重大变革，为以往难以解决的问题提供了全新解决方案。尽管生成式AI代理具备前所未有的能力，但也带来独特的安全风险，尤其在高风险环境中运行的自主机器更需健全的防护措施。本文研究了生成模型作为代理嵌入物理自主机器时不断演变的安全需求，并将其与低风险场景下的安全考量进行对比。探讨了确保生成式AI驱动的自主机器安全部署所面临的挑战与机遇。此外，展望了未来人工智能驱动的自主系统发展，强调评估和沟通安全风险的重要性。作为应对这些关切的重要步骤，我们建议开发并实施针对生成式AI技术在自主机器中应用的综合性安全评分卡。
原文摘要 · Abstract (English)
The integration of Generative Artificial Intelligence (AI) into autonomous machines represents a major paradigm shift in how these systems operate and unlocks new solutions to problems once deemed intractable. Although generative AI agents provide unparalleled capabilities, they also have unique safety concerns. These challenges require robust safeguards, especially for autonomous machines that operate in high-stakes environments. This work investigates the evolving safety requirements when generative models are integrated as agents into physical autonomous machines, comparing these to safety considerations in less critical AI applications. We explore the challenges and opportunities to ensure the safe deployment of generative AI-driven autonomous machines. Furthermore, we provide a forward-looking perspective on the future of AI-driven autonomous systems and emphasize the importance of evaluating and communicating safety risks. As an important step towards addressing these concerns, we recommend the development and implementation of comprehensive safety scorecards for the use of generative AI technologies in autonomous machines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。