用动作条件生成未来视觉，低成本评估机器人操作性能。
EnerVerse-AC: Envisioning Embodied Environments with Action Condition
- 基于预测动作生成动态多视角图像，实现可控仿真
- 通过失败轨迹扩充数据，提升模型泛化能力
- 无需真实机器人或复杂模拟，适合算法快速验证
机器人模仿学习已从静态任务发展到动态交互场景，但实时交互测试成本高、难度大。我们提出EnerVerse-AC（EVAC），一种动作条件世界模型，可根据代理预测动作生成未来视觉观测，实现真实且可控制的机器人推理。基于已有架构，EVAC引入多层级动作条件机制和射线图编码，支持动态多视图图像生成，并通过多样化失败轨迹扩充训练数据以增强泛化性。作为数据引擎与评估工具，EVAC将人类收集轨迹扩展为多样化数据集，生成逼真的动作条件视频观测用于策略测试，无需物理机器人或复杂仿真。该方法显著降低评估成本，同时保持高保真度。大量实验验证了其有效性。代码、模型权重及数据集详见 <https://annaj2178.github.io/EnerverseAC.github.io>。
原文摘要 · Abstract (English)
Robotic imitation learning has advanced from solving static tasks to addressing dynamic interaction scenarios, but testing and evaluation remain costly and challenging due to the need for real-time interaction with dynamic environments. We propose EnerVerse-AC (EVAC), an action-conditional world model that generates future visual observations based on an agent's predicted actions, enabling realistic and controllable robotic inference. Building on prior architectures, EVAC introduces a multi-level action-conditioning mechanism and ray map encoding for dynamic multi-view image generation while expanding training data with diverse failure trajectories to improve generalization. As both a data engine and evaluator, EVAC augments human-collected trajectories into diverse datasets and generates realistic, action-conditioned video observations for policy testing, eliminating the need for physical robots or complex simulations. This approach significantly reduces costs while maintaining high fidelity in robotic manipulation evaluation. Extensive experiments validate the effectiveness of our method. Code, checkpoints, and datasets can be found at <https://annaj2178.github.io/EnerverseAC.github.io>.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。