arXiv:2512.03399cs.LG2025-12被引 10

解决AI与机构价值观对齐难题,提出厚价值模型新框架

Full-Stack Alignment: Co-Aligning AI and Institutions with Thick Models of Value

  • 用厚价值模型统一表示个人与集体价值,区分持久信念与临时偏好
  • 在五类场景中验证可实现价值共担、规范推理与社会嵌入决策
  • 适合关注伦理对齐、制度设计与多主体协作的研究者

有益的社会结果无法仅通过将单个AI系统与操作者或用户意图对齐来保证。即使某AI完全对齐其所属组织的意图,若该组织目标与其他机构及个体的价值观不一致,仍可能导致不良后果。因此需要全栈对齐——同时对齐AI系统及其所塑造的制度与人类共同价值。此过程无需预设特定的个人或集体繁荣愿景。现有价值表征方式(如效用函数、偏好排序、无结构文本)难以有效应对这一挑战:它们难以区分价值与其它信号,缺乏规范性推理能力,且无法建模集体利益。我们提出需采用厚价值模型,通过结构化方式呈现价值与规范,使系统能识别持久价值、建模个体选择的社会背景,并进行规范性推理,适用于新领域。我们在五个方向验证该方法:AI价值托管、具备规范能力的智能体、双赢谈判系统、意义保真经济机制以及民主监管制度。

原文摘要 · Abstract (English)

Beneficial societal outcomes cannot be guaranteed by aligning individual AI systems with the intentions of their operators or users. Even an AI system that is perfectly aligned to the intentions of its operating organization can lead to bad outcomes if the goals of that organization are misaligned with those of other institutions and individuals. For this reason, we need full-stack alignment, the concurrent alignment of AI systems and the institutions that shape them with what people value. This can be done without imposing a particular vision of individual or collective flourishing. We argue that current approaches for representing values, such as utility functions, preference orderings, or unstructured text, struggle to address these and other issues effectively. They struggle to distinguish values from other signals, to support principled normative reasoning, and to model collective goods. We propose thick models of value will be needed. These structure the way values and norms are represented, enabling systems to distinguish enduring values from fleeting preferences, to model the social embedding of individual choices, and to reason normatively, applying values in new domains. We demonstrate this approach in five areas: AI value stewardship, normatively competent agents, win-win negotiation systems, meaning-preserving economic mechanisms, and democratic regulatory institutions.

价值对齐制度设计规范推理厚价值模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。