arXiv:2607.24649cs.AI2026-07

检验大模型模拟人类行为时理由是否真实可靠

Reason-Mediated Behavioral Models for Auditing LLM Social Simulators

论文配图:Reason-Mediated Behavioral Models for Auditing LLM Social Simulators
图 1 · 摘自论文原文
  • 用符号化理由状态分析人类决策逻辑
  • 人类理由能显著提升购买意向预测准确率
  • 大模型常模仿概念描述而非真实决策路径

大语言模型越来越多地被用作社会模拟器,包括作为合成问卷受访者。现有评估主要关注模拟结果是否接近人类结果,但我们认为这不够充分:模拟器可能匹配最终答案,但使用错误的推理模式。我们通过一项包含94人的防晒产品概念测试研究该问题,每位受访者评估三个产品概念并写出开放式理由。我们将这些理由映射为带符号的理由状态 $Z$,正号支持采纳,负号阻碍采纳。该方法提供实用审计:在固定受访者特征 $D$、品类背景 $K$ 和概念处理 $X$ 的条件下,人类理由能否显著提升对行为 $Y$ 的预测?大模型能否在未见人类理由或结果的情况下模拟出相同理由状态?结果显示,人类理由显著提升对购买意图的预测性能;而大模型生成的理由虽看似合理,却往往重复概念说明,难以还原真实的接受或拒绝路径。本文提出一种社会模拟器的评估框架。理由状态本身不直接揭示因果效应,但可作为可解释测试,检验模拟器理由是否与人类证据一致。

原文摘要 · Abstract (English)

Large language models are increasingly used as social simulators, including as synthetic survey respondents. Most evaluations ask whether simulated outcomes resemble human outcomes. We argue that this is necessary but too weak: a simulator can match the final answer while using the wrong rationale-derived reason pattern. We study this problem through a 94-person sunscreen concept test in which each respondent evaluated three product concepts and wrote open-ended rationales. We map those rationales into signed reason states $Z$, where positive signs support adoption and negative signs block it. This gives a practical audit: holding respondent descriptors $D$, category context $K$, and concept treatment $X$ fixed, do human rationale-derived reasons help predict behavior $Y$, and can an LLM simulate the same reason state without seeing the human rationale or outcome? Human rationale-derived reasons substantially improve held-out prediction of purchase intent. LLM-simulated reasons are more brittle: they often sound plausible, but frequently echo the concept board rather than recover the respondent's acceptance or rejection path. The paper contributes an evaluation framework for social simulators. Reason states do not identify natural causal effects by themselves, but they provide an interpretable test of whether a simulator's stated reasons align with human evidence.

大模型评估社会模拟可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。