用心理文化框架提升大模型对不同国家群体偏好的建模能力。
PALMs: Using Multi Construct-Grounded Rationales for Modeling Population Preferences in LLMs

- 基于心理与文化构念生成推理理由,作为隐式监督信号进行对齐训练。
- 在五国群体偏好建模中平均比最优基线提升8.59%。
- 适合需要跨文化群体模拟的研究者和应用开发者。
大语言模型被广泛用于模拟个体行为,但真实反映群体特征需捕捉价值观、信念和文化规范的系统性差异。本文提出针对五个国家(美国、印度、巴西、法国、意大利)的群体对齐语言模型(PALMs),通过融合心理学与文化构念生成推理理由,并将其作为潜在监督信号,在偏好调优中实现群体特异性对齐。在人格、价值观与信念、文化规范、道德四个维度评估中,PALMs持续优于基线模型,包括专门针对文化的模型,平均相对提升达8.59%。值得注意的是,基于构念的推理理由优于仅依赖人口统计提示或调查微调的方法,表明心理与文化基础提供比表面响应分布更丰富的归纳信号。进一步验证其在下游任务中的强泛化能力:在个性化奖励建模中超越最佳基线5.19%,在群体模拟中提升6.34%,并在社会推理任务中表现优异。数据集与代码已开源:https://github.com/limenlp/PALMs。
原文摘要 · Abstract (English)
Large language models are being extensively used to simulate individual user behavior, yet faithfully representing a population requires capturing the systematic variation in values, beliefs, and cultural norms that distinguish one group from another. We introduce Population Aligned Language Models (PALMs), a suite of models each aligned to specific populations, covering five countries: USA, India, Brazil, France and Italy. PALMs are created by synthesizing rationales grounded in psychological and cultural constructs and using these as latent supervision during preference tuning for population-specific alignment. Evaluated across four dimensions: personality, values and beliefs, cultural norms, and morality, PALMs consistently outperform baselines, including culture-specialized models, achieving an average of 8.59% relative improvement over the best baseline across all five populations. Notably, construct-grounded rationales outperform both demographic prompting and survey-based fine-tuning, suggesting that grounding preference learning in psychology and culture provides a richer inductive signal than surface-level response distributions. We further demonstrate strong generalization to downstream applications with- out task-specific supervision: outperforming best baselines by 5.19% in personalized reward modeling, 6.34% in population simulation, and showing strong transfer to social reasoning tasks. Datasets and code are available at: https://github.com/limenlp/PALMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。