提出可扩展的多智能体世界模型,支持多人实时交互生成。
Gamma-World: Generative Multi-Agent World Modeling Beyond Two Players

- 用单纯形旋转编码实现无序可扩展的智能体身份表示
- 通过稀疏枢纽注意力将跨智能体计算从二次降至线性
- 实现实时24帧生成,支持2至4人场景泛化
交互式视频生成的世界模型长期局限于单智能体设定,即未来观测由单一控制信号生成。然而,许多生成环境需要多智能体互动:多个玩家、机器人或具身智能体在共享空间中同时行动。将世界模型扩展到此类设定需遵循原则性的多智能体设计:智能体应保持独立可控、排列对称,并支持高效推理,同时保证时间与视角间的一致性。本文提出一种生成式多智能体世界模型,引入Simplex Rotary Agent Encoding——3D RoPE的无参数扩展,将智能体表示为旋转角度空间中的正单纯形顶点,使每个智能体具有独特相位的同时保持排列等价性,实现无需学习槽位身份或固定顺序的可扩展身份表示。为避免智能体间密集全连接注意力,进一步提出Sparse Hub Attention,通过可学习的枢纽令牌中介跨智能体交互,将跨智能体注意力成本从二次方降至线性。为实现实时回放,将全上下文扩散教师模型蒸馏为因果学生模型,通过KV缓存逐块序列生成,实现24帧每秒的动作响应生成。在多人虚拟环境实验中,模型在视频保真度、动作可控性和智能体间一致性上优于基于槽位和密集注意力的基线,且无需额外训练即可从两人推广至四人场景。
原文摘要 · Abstract (English)
World models for interactive video generation have largely focused on single-agent settings, where future observations are generated from a single control signal. However, many generated environments require multi-agent interaction: multiple players, robots, or embodied agents act simultaneously within a shared space. Scaling world models to such settings requires a principled multi-agent design: agents should remain independently controllable, permutation-symmetric, and support efficient inference while maintaining consistency across time and perspectives. In this paper, we present our generative multi-agent world model for interactive simulation. It introduces Simplex Rotary Agent Encoding, a parameter-free extension of 3D RoPE that represents agents as vertices of a regular simplex in rotary angle space. This gives each agent a distinct phase while making all agents permutation-equivalent, enabling scalable agent identity without learned per-slot identities or a fixed agent ordering. To avoid dense all-to-all attention across agents, we further propose Sparse Hub Attention, where learnable hub tokens mediate token interaction across agents, reducing cross-agent attention cost from quadratic to linear in the number of agents. For real-time rollout, we distill a full-context diffusion teacher into a causal student that generates temporal blocks sequentially with KV caching, enabling action-responsive generation at 24 FPS. Experiments in multiplayer virtual environments show that our model improves video fidelity, action controllability, and inter-agent consistency over slot-based and dense-attention baselines, while generalizing from two to four players without additional training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。