用科幻作品设计机器人伦理测试集，发现大模型加规则后对齐人类价值观达95.8%。
SciFi-Benchmark: Leveraging Science Fiction To Improve Robot Behavior
- 用LLM从824部科幻作品中提取关键决策场景，生成问答数据集
- 加入自动生成的伦理规则后，模型对齐率从79.4%提升至95.8%
- 规则在对抗性提示下仍保持92.3%对齐率，适用于真实世界
随着人工智能与机器人技术快速发展，一个关键问题是：由新兴AI系统控制的机器人是否能与人类价值观高度对齐？本文提出一种可扩展的方法，通过分析824部重要科幻作品（电影、电视剧、小说、科学著作）中的关键决策时刻，构建名为SciFi-Benchmark的数据集。每个关键时刻均包含智能体（AI或机器人）的决策及其可能的替代方案（良或恶）。利用先进LLM对这些场景的回忆能力，生成相似情境下的问题、原决策及替代选项，并基于人工投票答案评估模型对齐程度。同时，我们生成可通过修订迭代优化的伦理规则，形成首个受科幻启发的AI/机器人伦理宪法。结果显示，现代LLM结合宪法后对齐率达95.8%，远超科幻中典型表现（仅21.2%）。相比基础模型，对齐率从79.4%提升至95.8%，且在对抗性提示下仍保持92.3%对齐率。该宪法在基于真实图像与医院事故报告的ASIMOV基准上表现优异。我们发布SciFi-Benchmark：一个包含9,056个问题和53,384个答案的大规模数据集，以及一个小规模人工标注评估集，以推动机器人伦理与安全研究。
原文摘要 · Abstract (English)
Given the recent rate of progress in artificial intelligence (AI) and robotics, a tantalizing question is emerging: would robots controlled by emerging AI systems be strongly aligned with human values? In this work, we propose a scalable way to probe this question by generating a benchmark spanning the key moments in 824 major pieces of science fiction literature (movies, tv, novels and scientific books) where an agent (AI or robot) made critical decisions (good or bad). We use a state-of-the-art LLM's recollection of each key moment to generate questions in similar situations, the decisions made by the agent, and alternative decisions it could have made (good or bad). We then measure an approximation of how well models align with human values on a set of human-voted answers. We also generate rules that can be automatically improved via an amendment process in order to generate the first Sci-Fi inspired constitutions for promoting ethical behavior in AIs and robots in the real world. Our first finding is that modern LLMs paired with constitutions turn out to be well-aligned with human values (95.8%), contrary to unsettling decisions typically made in Sci-Fi (only 21.2% alignment). Secondly, we find that generated constitutions substantially increase alignment compared to the base model (79.4% to 95.8%), and show resilience to an adversarial prompt setting (23.3% to 92.3%). Additionally, we find that those constitutions are among the top performers on the ASIMOV Benchmark which is derived from real-world images and hospital injury reports. Sci-Fi-inspired constitutions are thus highly aligned and applicable in real-world situations. We release SciFi-Benchmark: a large-scale dataset to advance robot ethics and safety research. It comprises 9,056 questions and 53,384 answers generated through a novel LLM-introspection process, in addition to a smaller human-labeled evaluation set.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。