arXiv:2411.06790cs.CYcs.CL2024-11被引 17

测试52个大模型在自动驾驶道德困境中的判断,发现模型越大越像人,但更新未必更好。

Large-scale moral machine experiment on large language models

  • 用联合分析法评估52个大模型的道德判断与人类偏好匹配度。
  • 超过100亿参数的模型更接近人类判断,且参数量越大偏差越小。
  • 模型更新不必然提升道德判断,需关注文化背景和计算效率。

大型语言模型(LLMs)的快速演进及其在自动驾驶系统中的潜在应用,要求我们理解其道德决策能力。此前研究仅考察了4个主流大模型,而当前模型生态变化迅速,亟需更全面分析。本文基于道德机器实验框架,评估了52个不同大模型(包括GPT、Claude、Gemini等专有模型及Llama、Gemma等开源模型)在自动驾驶伦理困境中的判断表现,采用联合分析法衡量其与人类道德偏好的一致性,并探究模型规模、版本更新与架构的影响。结果表明,专有模型和参数量超过100亿的开源模型表现出与人类判断更接近的倾向,且开源模型中模型规模与人类判断距离呈显著负相关。然而,模型更新并未持续提升对人类偏好的拟合度,许多模型对特定伦理原则存在过度强调。这说明尽管增大模型规模可能自然提升道德判断的人类相似性,但在实际部署中仍需权衡判断质量与计算效率。本研究为自动驾驶系统的伦理设计提供关键依据,并强调文化语境在人工智能道德决策中的重要性。

原文摘要 · Abstract (English)

The rapid advancement of Large Language Models (LLMs) and their potential integration into autonomous driving systems necessitates understanding their moral decision-making capabilities. While our previous study examined four prominent LLMs using the Moral Machine experimental framework, the dynamic landscape of LLM development demands a more comprehensive analysis. Here, we evaluate moral judgments across 52 different LLMs, including multiple versions of proprietary models (GPT, Claude, Gemini) and open-source alternatives (Llama, Gemma), to assess their alignment with human moral preferences in autonomous driving scenarios. Using a conjoint analysis framework, we evaluated how closely LLM responses aligned with human preferences in ethical dilemmas and examined the effects of model size, updates, and architecture. Results showed that proprietary models and open-source models exceeding 10 billion parameters demonstrated relatively close alignment with human judgments, with a significant negative correlation between model size and distance from human judgments in open-source models. However, model updates did not consistently improve alignment with human preferences, and many LLMs showed excessive emphasis on specific ethical principles. These findings suggest that while increasing model size may naturally lead to more human-like moral judgments, practical implementation in autonomous driving systems requires careful consideration of the trade-off between judgment quality and computational efficiency. Our comprehensive analysis provides crucial insights for the ethical design of autonomous systems and highlights the importance of considering cultural contexts in AI moral decision-making.

大模型道德判断自动驾驶伦理设计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。