arXiv:2501.02531cs.CYcs.CL2025-01被引 1

对比人类与大模型对通用人工智能的情感,发现模型态度更积极且存在偏见。

Towards New Benchmark for AI Alignment & Sentiment Analysis in Socially Important Issues: A Comparative Study of Human and LLMs in the Context of AGI

  • 用李克特量表比较7个大模型与3组人类对AGI的情感
  • 模型情感分3.32~4.12,人类平均仅2.97,GPT-4最乐观
  • 提出跨时序、多语言的SAAS-AI基准,助力政策制定

随着通用人工智能系统日益融入社会,在信息获取、内容生成、问题解决、文本分析、编程及流程运行等方面发挥作用,评估其长期影响至关重要。本研究通过李克特量表调查,探讨大语言模型(LLMs)与人类对人工通用智能(AGI)的情感倾向。分析了包括GPT-4和Bard在内的7个大模型,并与三组独立人类样本数据进行对比,评估了连续三天内情感的时序变化。结果显示,大模型间情感评分存在差异,范围为3.32至4.12(满分5分),其中GPT-4表现最积极,Bard则趋向中性;而人类样本平均情感得分仅为2.97。分析揭示了大模型情感形成中潜在的利益冲突与偏见,提示其可能微妙影响社会认知。为应对监管监督与文化适配评估需求,本文提出社会型人工智能对齐与情感基准(SAAS-AI),采用多维度提示与经实证验证的社会价值框架,从时间、模型与多语言维度评估语言模型输出。该基准旨在为政策制定者与AI机构(如欧盟人工智能法案框架下)提供可靠、可操作的洞察,支持人工智能在国家与国际层面与人类价值观、公众情绪及伦理规范的对齐。未来研究需进一步优化SAAS-AI的操作化路径,并通过系统性实证测试评估其有效性。

原文摘要 · Abstract (English)

As general-purpose artificial intelligence systems become increasingly integrated into society and are used for information seeking, content generation, problem solving, textual analysis, coding, and running processes, it is crucial to assess their long-term impact on humans. This research explores the sentiment of large language models (LLMs) and humans toward artificial general intelligence (AGI) using a Likert-scale survey. Seven LLMs, including GPT-4 and Bard, were analyzed and compared with sentiment data from three independent human sample populations. Temporal variations in sentiment were also evaluated over three consecutive days. The results show a diversity in sentiment scores among LLMs, ranging from 3.32 to 4.12 out of 5. GPT-4 recorded the most positive sentiment toward AGI, while Bard leaned toward a neutral sentiment. In contrast, the human samples showed a lower average sentiment of 2.97. The analysis outlines potential conflicts of interest and biases in the sentiment formation of LLMs, and indicates that LLMs could subtly influence societal perceptions. To address the need for regulatory oversight and culturally grounded assessments of AI systems, we introduce the Societal AI Alignment and Sentiment Benchmark (SAAS-AI), which leverages multidimensional prompts and empirically validated societal value frameworks to evaluate language model outputs across temporal, model, and multilingual axes. This benchmark is designed to guide policymakers and AI agencies, including within frameworks such as the EU AI Act, by providing robust, actionable insights into AI alignment with human values, public sentiment, and ethical norms at both national and international levels. Future research should further refine the operationalization of the SAAS-AI benchmark and systematically evaluate its effectiveness through comprehensive empirical testing.

AI对齐情感分析大模型评估政策基准

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。