用一个策略让上千个角色有个性且实时可控，效果远超传统方法。
One Policy, Infinite NPCs: Persona-Traceable Shared RL Policies for Scalable Game Agents
- 用大模型嵌入固化角色人格，通过低秩投影实现高效共享控制。
- 零样本识别准确率是随机水平的17倍，行为与人格匹配度达0.73。
- 适合需要大量高拟真角色的开放世界游戏开发，性能可商用部署。
在包含300个角色的人格模拟基准测试中，pcsp实现了组合式零样本人格识别，准确率最高达随机水平的17倍，人格-行为语义对齐度Spearman rho约为0.73，推理速度比基于LLM的基线快22倍。生命模拟类游戏需要数百至数千个非玩家角色（NPC），其行为需与各自人格一致，同时支持设计师通过自然语言控制。现有方法在人格一致性、可控性或实时推理上存在瓶颈。本文提出pcsp（Persona Conditioned Shared Policy），一种基于冻结的自由描述人格大模型嵌入的单一强化学习策略。pcsp结合每个角色一次性编码人格、低秩人格投影、神经人格条件化及PPO + InfoNCE一致性 + KL多样性联合训练目标。三种实验设置下的消融实验表明，InfoNCE轨迹一致性目标至关重要：移除后零样本人格识别性能退化至随机水平。在Melting Pot 2.4.0子任务上的外部验证确认，该方法可在多智能体战略环境中生成人格驱动的行为差异。本文区分两种留出评估方式：组合式零样本与词汇扩展留出。最终，基于UE5的部署在64个代理场景下复现了引擎内的人格条件化消融实验，失败率低，证明其亚帧级推理能力可在商业游戏引擎中稳定运行。结果表明，共享强化学习策略可实现可扩展、实时、人格化的NPC控制。
原文摘要 · Abstract (English)
On a 300-persona life-simulation benchmark, pcsp achieves compositional zero-shot persona identification up to 17x above chance, Spearman rho approx 0.73 semantic-behavioral alignment, and 22x faster inference than an LLM-as-policy baseline. Life simulation games require hundreds to thousands of non-player characters (NPCs) that behave consistently with distinct personalities while remaining controllable through designer-authored natural language. Existing methods fail on constraints like persona consistency, controllability, or real-time inference. We introduce pcsp (Persona Conditioned Shared Policy), a single reinforcement learning policy conditioned on frozen LLM embeddings of free-form persona descriptions. pcsp combines once-per-NPC persona encoding, low-rank persona projection, neural persona conditioning, and a PPO + InfoNCE consistency + KL diversity training objective. Across three experimental settings, ablations show that the InfoNCE trajectory-consistency objective is load bearing: removing it collapses zero-shot persona identification to chance. External validation on Melting Pot 2.4.0 substrates confirms that our method produces persona-conditioned behavioral divergence in multi-agent strategic environments. We distinguish two senses of held-out evaluation: compositional zero-shot and vocabulary-expansion held-out. Finally, a UE5 deployment reproduces the in-engine persona-conditioning ablation at 64 agents with a low failure rate, showing that the sub-frame inference profile survives in a commercial game engine. These results prove that shared RL policies can support scalable, real-time, persona-conditioned NPC control.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。