用大模型嵌入还原人类价值观结构,效果媲美人工调查。
Neural network embeddings recover value dimensions from psychometric survey items on par with human data
- 用SQuID方法处理语言模型嵌入,自动捕捉价值维度关系。
- 解释了55%的人类判断中维度相似性的方差,跨国数据对齐良好。
- 无需领域微调,可推广至多种人格量表,适合快速构建心理测量工具。
我们证明,通过“问卷项目嵌入差异”(SQuID)处理的大语言模型嵌入,能够恢复基于人类评分的修订版人物价值问卷(PVQ-RR)所揭示的人类价值观结构。在内部一致性、维度相关性及多维尺度配置等指标上对比多种嵌入模型。与以往方法不同,SQuID无需领域特定微调或标注重注,即可获得维度间的负相关。定量分析显示,该方法解释了55%的维度-维度相似性方差,多维尺度配置与来自49个国家的汇总人类数据高度一致。在IPIP、BFI-2、HEXACO三个个性量表上的泛化测试表明,SQuID持续提升相关性范围,表明其适用性超越价值理论。结果表明,语义嵌入可有效复现传统人工调查建立的心理测量结构。该方法在成本、可扩展性和灵活性上具有显著优势,同时保持与传统方法相当的质量。研究对心理测量学与社会科学具有重要意义,提供了一种可扩展的互补测量方法。
原文摘要 · Abstract (English)
We demonstrate that embeddings derived from large language models, when processed with "Survey and Questionnaire Item Embeddings Differentials" (SQuID), can recover the structure of human values obtained from human rater judgments on the Revised Portrait Value Questionnaire (PVQ-RR). We compare multiple embedding models across a number of evaluation metrics including internal consistency, dimension correlations and multidimensional scaling configurations. Unlike previous approaches, SQuID addresses the challenge of obtaining negative correlations between dimensions without requiring domain-specific fine-tuning or training data re-annotation. Quantitative analysis reveals that our embedding-based approach explains 55% of variance in dimension-dimension similarities compared to human data. Multidimensional scaling configurations show alignment with pooled human data from 49 different countries. Generalizability tests across three personality inventories (IPIP, BFI-2, HEXACO) demonstrate that SQuID consistently increases correlation ranges, suggesting applicability beyond value theory. These results show that semantic embeddings can effectively replicate psychometric structures previously established through extensive human surveys. The approach offers substantial advantages in cost, scalability and flexibility while maintaining comparable quality to traditional methods. Our findings have significant implications for psychometrics and social science research, providing a complementary methodology that could expand the scope of human behavior and experience represented in measurement tools.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。