用亲和力强化学习让AI在恋爱游戏中既竞争又合作，行为更像人。
Fog of Love: Engineering Virtuous Agent Behavior with Affinity-based Reinforcement Learning in a Game Environment
- 基于亲和力设计奖励正则化，引导AI自主做出道德选择。
- 在复杂双人博弈中，亲和力策略使得分显著提升。
- 结果可解释性强，适合研究伦理智能与人机协作。
让人工智能具备美德行为受到越来越多关注。一种方法是亲和力增强的强化学习，通过在目标函数中加入策略正则化，激励美德行为而不完全依赖奖励设计。此前该方法仅在状态和动作空间极小的网格世界和简单问题中验证有效。为拓展至更复杂环境，本文引入基于桌面角色扮演游戏《迷雾之爱》的双人多智能体环境。两名智能体需在追求个人美德的同时维系关系。由于多智能体特性，传统多智能体深度确定性策略梯度算法无法有效实现竞争或合作。实验表明，局部亲和力机制显著提升智能体在竞争与合作两方面表现，带来更高综合得分。这不仅促成美德行为，还明确其行为目的，使决策过程达到人类可理解水平。
原文摘要 · Abstract (English)
Instilling virtuous behavior in artificial intelligence has seen increasing interest. One of the techniques proposed is known as affinity-based reinforcement learning, which uses policy regularization on the objective function to incentivize virtuous actions without being fully dependent on the reward function design. Thus far, this technique has been demonstrated to be effective in grid worlds and toy-problem environments with minimal state and action spaces. To expand this research to more sophisticated environments, we introduce a two-player multi-agent environment based on the role-playing board game known as Fog of Love. In this environment, two agents compete to fulfill their individual virtues, while also cooperating to satisfy their relationship. Given the multi-agent nature, this is a complex problem where multi-agent deep deterministic policy gradient agents neither compete nor cooperate successfully. We present evidence that localized affinities enhance agent performance in achieving both competitive and cooperative objectives, resulting from superior overall scores in both domains. This not only results in virtuous choices but also clarifies an agent's teleology and makes its behavior human-level interpretable.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。