指令微调模型无法随机采样，只能输出固定答案。
Instruction-Tuned Language Models Cannot Sample from Distributions They Can Describe
- 模型对同一问题重复输出相同答案，超过一半项一致
- 内部概率高度集中于单一选项，分布采样能力崩溃
- 可描述分布但无法采样，适合用于生成真实人群响应的模拟
硅基采样将语言模型视为人类调查受访者，每次调用视为从角色响应分布中独立抽样。我们发现这种抽样并不存在：指令微调模型不会从分布中采样，而是坍缩为单一输出。在同一公共意见基准上，同一角色对同一问题的回答在超过半数项目中重复；模型内部概率也集中于单一选项。该失败与指令后训练相关且被放大：指令微调模型在所有可比较家族中均劣于其基础模型，差距随每轮后训练阶段、微调更新量增大而加剧，且在非指令数据继续预训练下无改善。然而知识仍存在：同一模型虽无法采样，却可在单次调用中准确描述分布。我们称此为KNOWS/DOES分裂。利用该分裂，仅需一次描述性调用即可使误差较角色聚合降低超过一半；当需要个性化输出时，我们提出提示扰动Argyle（PPA），将误差再降低21%，无需额外成本即可模拟真实人群差异。
原文摘要 · Abstract (English)
Silicon sampling uses language models as proxies for human survey respondents, treating each model call as an independent draw from the persona's response distribution. We show this draw does not exist: instruction-tuned models do not sample from distributions, they collapse to a single output. The same persona on the same question returns the same answer on more than half of items in a public-opinion benchmark, and the model's internal probabilities concentrate on a single option. The failure is associated with, and amplified by, instruction-targeted post-training: instruction-tuned models are worse than their own bases in every family we can compare, the gap widens at each successive post-training stage and with the size of the tuning update, and continued pretraining on non-instruction tokens leaves it unchanged. Yet the knowledge survives: the same model that cannot sample from a distribution can describe it accurately in a single call. We call this gap the KNOWS/DOES split. Exploiting the split, a single call that asks the model to describe the response distribution more than halves the error against human survey data compared to persona aggregation. When per-persona outputs are required, we propose Prompt-Perturbed Argyle (PPA), which reduces the same error by 21\%, spreading each persona's answers to mirror real population differences at no added cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。