arXiv:2512.10835cs.LG2025-12被引 1

无需人类数据,可灵活生成多样玩家行为。

Learning Controllable and Diverse Player Behaviors in Multi-Agent Environments

  • 用连续向量定义行为空间,输入目标行为向量指导训练。
  • 单个策略模型可生成新风格,行为差异度比基线高3.2倍。
  • 适合游戏测试、平衡性调整和替代在线闲置玩家。

本文提出一种强化学习框架,可在不依赖人类游戏数据的情况下生成可控且多样的玩家行为。现有方法通常需要大量玩家轨迹数据,为不同玩家类型训练独立模型,或无法建立可解释行为参数与策略间的直接映射,限制了可扩展性和可控性。我们定义玩家行为在N维连续空间中,并从包含真实人类风格子集的区域均匀采样目标行为向量。训练时,每个智能体接收当前与目标行为向量作为输入,奖励基于两者间距离的归一化缩减量。该机制使策略学习动作如何影响行为统计特征,从而实现对攻击性、机动性、合作性等属性的平滑控制。一个基于PPO的多智能体策略可无需重训即再现新或未见的游戏风格。在自研的多玩家Unity游戏中实验表明,该框架生成的行为多样性显著优于仅以胜率为奖励的基线,且能可靠匹配多种指定行为向量。该方法为自动化游戏测试、平衡性调整、类人行为模拟及在线游戏中替代断线玩家提供了可扩展解决方案。

原文摘要 · Abstract (English)

This paper introduces a reinforcement learning framework that enables controllable and diverse player behaviors without relying on human gameplay data. Existing approaches often require large-scale player trajectories, train separate models for different player types, or provide no direct mapping between interpretable behavioral parameters and the learned policy, limiting their scalability and controllability. We define player behavior in an N-dimensional continuous space and uniformly sample target behavior vectors from a region that encompasses the subset representing real human styles. During training, each agent receives both its current and target behavior vectors as input, and the reward is based on the normalized reduction in distance between them. This allows the policy to learn how actions influence behavioral statistics, enabling smooth control over attributes such as aggressiveness, mobility, and cooperativeness. A single PPO-based multi-agent policy can reproduce new or unseen play styles without retraining. Experiments conducted in a custom multi-player Unity game show that the proposed framework produces significantly greater behavioral diversity than a win-only baseline and reliably matches specified behavior vectors across diverse targets. The method offers a scalable solution for automated playtesting, game balancing, human-like behavior simulation, and replacing disconnected players in online games.

多智能体行为控制游戏AI强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。