用生成式投射测试提升大模型心理评估的可靠性
GenPT: Beyond Self-Report for Reliable LLM Psychometrics via Generative Projective Testing

- 用生成新刺激重构投射测验,构建三阶段评估流程
- 在社会期待框架下,传统问卷出现系统性偏差,而该方法保持稳定
- 适合需要抗污染、防偏见的心理状态评估场景
自报告问卷仍是探测角色驱动智能体(PC-Agents)心理状态的主要工具,但存在训练语料污染和社交期望偏差两大问题。为克服这些方法论瓶颈,我们探索将投射范式转化为可靠的测量工具。提出「GenPT」(生成式投射测试),通过新生成的刺激重构塔维斯-罗夏墨迹测验(TAT)、罗夏测验(Rorschach)和主题统觉测验(SCT),并建立三阶段流程以提取标准化心理指标。在CharacterRAG与AnnaAgent生成的PC-Agents上评估,结果表明:在社交期望框架下,传统问卷在自杀意念维度出现显著系统性偏移;而GenPT收集的行为模式始终接近对称基线。在纵向咨询场景中,以Qwen3为骨干模型时,GenPT对抑郁水平的评估变化幅度比问卷高出约一个数量级。整体表明,GenPT在需抵抗污染、规避偏倚、敏感应对情境的场景中,可有效补充自报告方法。代码与刺激材料见https://github.com/sci-m-wang/GenPT。
原文摘要 · Abstract (English)
Self-report questionnaires remain the prevailing tool for probing the psychological states of persona-conditioned agents (PC-Agents). However, classical instruments inherit two well-known threats: contamination from training corpora and directional bias driven by social-desirability or contextual framing. To overcome these methodological bottlenecks, we ask whether projective paradigms can be adapted into a robust psychometric tool. We introduce \textbf{GenPT} (Generative Projective Testing), which reformulates TAT, Rorschach, and SCT with newly generated stimuli and organizes assessment as a three-stage pipeline to derive standardized psychological indicators and target states. Evaluating PC-Agents induced via CharacterRAG and AnnaAgent profiles, we benchmark GenPT's reliability and validity against classical questionnaires. The results indicate that questionnaires exhibit systematic directional shifts under social-desirability framing, most strongly on suicide ideation. In contrast, GenPT's collected behavioral patterns stay near the symmetric baseline. Furthermore, under a longitudinal counselling context, GenPT-based depression assessment shifts by roughly an order of magnitude more than the questionnaire counterpart when Qwen3 serves as the backbone. Overall, GenPT complements self-report methods in scenarios where contamination resistance, bias asymmetry, and context sensitivity matter. Code and stimuli can be found at https://github.com/sci-m-wang/GenPT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。