通过相似人物关系图检测大模型回答的一致性，提升模拟调查的可信度。
When Persona Simulations Are Informative: Graph-Structured Signals for Pluralistic Opinion Sensing
- 构建人物相似性图谱，用局部空间一致性衡量回答是否受角色影响
- 在PVQ-RR问卷中，选出10%的高信息量人物，显著提升价值结构还原度
- 无需人工标注，适合用于自动化筛选高质量合成受访者
角色条件化大语言模型（LLMs）被广泛用于跨领域模拟调查回应。然而，表面的回答差异可能源于模型固有偏见或采样噪声，而非真正的角色驱动。我们提出，当语义相近的角色在回应上呈现一致偏移时，这种差异才是有信息量的。为此，我们引入无监督诊断指标Persona-Conditioned Informativeness（PCI），通过将角色建模为相似性图，利用局部莫兰指数（Local Moran's I）量化局部空间一致性，从而提取紧凑的角色子集，无需使用构念标签。为在无外部人类基准下评估PCI，我们在57项修订版人格价值观问卷（PVQ-RR）上测试其恢复潜在价值结构的能力。验证性因子分析（CFA）显示，由PCI选出的10%角色子集，在整体构念还原度上显著优于基于响应稳定性或随机选择的方法。结果支持PCI作为调查生成流程中筛选合成受访者的一项内在原理性诊断工具。
原文摘要 · Abstract (English)
Persona-conditioned large language models (LLMs) are increasingly used to simulate survey responses across diverse domains. However, apparent response variation can reflect unconditioned model priors or token sampling noise rather than systematic persona conditioning. We argue that persona-conditioned variation is informative when semantically similar personas exhibit concordant response shifts. To operationalize this principle, we introduce Persona-Conditioned Informativeness (PCI), an unsupervised diagnostic metric that measures whether semantically similar personas deviate in concordant directions relative to item-level sample baselines. By modeling personas as a similarity graph, PCI uses Local Moran's I to quantify local spatial coherence and extract compact persona subsets without using construct labels. To evaluate PCI without external human benchmarks, we test its ability to recover established latent value structure using the 57-item Portrait Values Questionnaire-Revised (PVQ-RR). Confirmatory factor analysis (CFA) shows that a PCI-selected 10% subset substantially improves overall construct recovery relative to response-stability and random selection. These findings support PCI as a principled internal diagnostic for screening synthetic respondents in survey pipelines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。