arXiv:2504.12299cs.AIcs.CV2025-04被引 6

用逆动力学模型让游戏智能体精准复现专家轨迹,应对复杂环境中的不确定性。

Adapting a World Model for Trajectory Following in a 3D Game

  • 采用逆动力学模型结合不同编码器与策略头,实现轨迹追踪。
  • 在多样数据下GPT风格策略头表现最优,低数据时DINOv2+GPT更佳。
  • 适用于需要高精度轨迹复现的3D游戏智能体训练场景。

模仿学习通过利用专家知识训练智能体,能复现给定轨迹是其核心能力之一。在现代3D视频游戏等复杂环境中,分布偏移和随机性要求超越简单动作重放的鲁棒方法。本研究在《Bleeding Edge》这款现代3D游戏中,应用逆动力学模型(IDM),测试了多种编码器与策略头组合进行轨迹跟随。同时,探索了几种未来对齐策略以缓解由偶然不确定性及智能体不完美导致的分布偏移问题。通过测量参考轨迹与智能体轨迹间的偏差距离及首次显著偏离点,发现最优配置依赖于具体设置:在多样化数据下,从头训练的编码器搭配GPT风格策略头表现最佳;低数据条件下,DINOv2编码器配合GPT风格策略头效果最优;而当预训练于多样化场景后微调特定行为时,GPT与MLP风格策略头表现相当。

原文摘要 · Abstract (English)

Imitation learning is a powerful tool for training agents by leveraging expert knowledge, and being able to replicate a given trajectory is an integral part of it. In complex environments, like modern 3D video games, distribution shift and stochasticity necessitate robust approaches beyond simple action replay. In this study, we apply Inverse Dynamics Models (IDM) with different encoders and policy heads to trajectory following in a modern 3D video game -- Bleeding Edge. Additionally, we investigate several future alignment strategies that address the distribution shift caused by the aleatoric uncertainty and imperfections of the agent. We measure both the trajectory deviation distance and the first significant deviation point between the reference and the agent's trajectory and show that the optimal configuration depends on the chosen setting. Our results show that in a diverse data setting, a GPT-style policy head with an encoder trained from scratch performs the best, DINOv2 encoder with the GPT-style policy head gives the best results in the low data regime, and both GPT-style and MLP-style policy heads had comparable results when pre-trained on a diverse setting and fine-tuned for a specific behaviour setting.

模仿学习轨迹追踪3D游戏逆动力学

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。