让AI在规则冲突时自主协调决策，确保符合人类期望。
Requirements for Aligned, Dynamic Resolution of Conflicts in Operational Constraints
- 提出动态协调规则冲突的决策框架
- 需融合规范、实用与情境三类知识
- 适用于高风险自主系统如自动驾驶
已部署的自主AI系统常需在新奇或定义不明确的场景中评估多个可能的行为序列。尽管经过大量训练，这些系统仍会遇到无一选项能完全满足所有操作约束（如规程、规则、法律、规范和目标）的情况。为达成符合人类期望与价值观的目标，智能体必须超越已有策略，自行构建、评估并解释候选行为路径。这一过程需要超出先前策略训练范围的上下文“知识”。本文刻画了此类情境下智能体决策所需的要求，并识别出使决策对智能体目标稳健且与人类期望一致所需的知识类型。结合分析与实证案例研究，我们探讨智能体如何整合规范性、实践性和情境性理解，以在复杂真实环境中选择并执行更契合的行动方案。
原文摘要 · Abstract (English)
Deployed, autonomous AI systems must often evaluate multiple plausible courses of action (extended sequences of behavior) in novel or under-specified contexts. Despite extensive training, these systems will inevitably encounter scenarios where no available course of action fully satisfies all operational constraints (e.g., operating procedures, rules, laws, norms, and goals). To achieve goals in accordance with human expectations and values, agents must go beyond their trained policies and instead construct, evaluate, and justify candidate courses of action. These processes require contextual "knowledge" that may lie outside prior (policy) training. This paper characterizes requirements for agent decision making in these contexts. It also identifies the types of knowledge agents require to make decisions robust to agent goals and aligned with human expectations. Drawing on both analysis and empirical case studies, we examine how agents need to integrate normative, pragmatic, and situational understanding to select and then to pursue more aligned courses of action in complex, real-world environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。