用大模型分析文本，自动测量人类与AI的价值观。
Measuring Human and AI Values Based on Generative Psychometrics with Large Language Models
- 将非结构化文本转化为可测的心理感知，动态评估价值观。
- 在博客数据上验证了方法稳定有效，优于传统心理测量工具。
- 首次实现对大模型价值观的上下文敏感测量，揭示安全关联性。
人类价值观及其测量是长期存在的跨学科课题。近期人工智能进展推动了该领域的复兴，大语言模型（LLMs）既成为测量工具，也成为测量对象。本文提出基于生成心理学的大模型价值测量框架（GPV），理论基础为文本揭示的选择性认知。核心思想是将非结构化文本动态解析为类似传统心理测量中的静态刺激，测量其反映的价值取向，并聚合结果。将GPV应用于人类撰写的博客，证明其具有稳定性、有效性，且优于现有心理测量工具。进一步将GPV扩展至大模型价值观测量，实现了三方面突破：1）基于大模型可扩展自由输出的可量化解析方法，支持情境化测量；2）对比分析多种测量范式，揭示先前方法的响应偏差；3）探索大模型价值观与其安全性之间的关联，发现不同价值体系的预测能力及各类价值对安全的影响。通过跨学科协作，旨在利用人工智能推动新一代心理测量学发展，同时以心理测量学助力价值对齐的人工智能。
原文摘要 · Abstract (English)
Human values and their measurement are long-standing interdisciplinary inquiry. Recent advances in AI have sparked renewed interest in this area, with large language models (LLMs) emerging as both tools and subjects of value measurement. This work introduces Generative Psychometrics for Values (GPV), an LLM-based, data-driven value measurement paradigm, theoretically grounded in text-revealed selective perceptions. The core idea is to dynamically parse unstructured texts into perceptions akin to static stimuli in traditional psychometrics, measure the value orientations they reveal, and aggregate the results. Applying GPV to human-authored blogs, we demonstrate its stability, validity, and superiority over prior psychological tools. Then, extending GPV to LLM value measurement, we advance the current art with 1) a psychometric methodology that measures LLM values based on their scalable and free-form outputs, enabling context-specific measurement; 2) a comparative analysis of measurement paradigms, indicating response biases of prior methods; and 3) an attempt to bridge LLM values and their safety, revealing the predictive power of different value systems and the impacts of various values on LLM safety. Through interdisciplinary efforts, we aim to leverage AI for next-generation psychometrics and psychometrics for value-aligned AI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。