考虑情境因素，重新设计道德评价聚合方式
Accounting for Context: Shaping Moral Credences for Value Alignment
- 在道德不确定性下引入情境因素建模决策
- 发现传统弱帕累托原则在现实情境中不成立
- 适合研究价值对齐与道德推理的学者
确保智能体行为与人类道德价值对齐，必然面临如何应对社会乃至个体间多元道德视角的问题。现有道德不确定性研究提出公平民主的行动评价聚合机制，但本文指出，聚合时必须考虑情境因素。例如，后果主义假设智能体能准确判断其行为对世界的影响，而这一假设在真实场景中常不成立。为此，本文形式化了在道德不确定性下考虑情境因素的智能体决策机制，并表明看似合理的弱帕累托原则在此框架下被违反。我们论证该问题实为辛普森悖论的一种变体，揭示了忽略情境影响的聚合机制的局限性。
原文摘要 · Abstract (English)
Ensuring that agent behaviours are aligned with human moral values inevitably raises the problem of how to account for the plurality of moral perspectives that societies -- and even individuals -- typically adopt. Work on moral uncertainty proposes mechanisms to fairly and democratically aggregate evaluations of actions across different moral theories. However, this paper argues that one needs to account for contextual factors when aggregating moral evaluations. For example, consequentialist perspectives assume an ability to accurately determine how an agent's actions change the world; an assumption that often does not hold in real world settings. We, therefore, formalise agent decision making under moral uncertainty, while also accounting for these kinds of contextual factors. We thereby show that a seemingly commonsensical property -- the weak Pareto principle -- is violated. We argue that this apparent problem is, in fact, a variation of Simpson's paradox, and hence reveals the limitations of aggregation mechanisms that ignore the impact of contextual factors.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。