首个从像素端到端训练的稳定世界模型，仅用两个损失项即可实现高效预测。
LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels
- 采用双损失机制：预测下一时刻嵌入+高斯分布正则化，避免表示坍塌。
- 1500万参数在单卡几小时内训练完成，推理速度比基础模型快48倍。
- 适用于2D/3D控制任务，还能捕捉物理结构并识别违反物理规律的事件。
联合嵌入预测架构（JEPAs）为在紧凑潜在空间中学习世界模型提供了有力框架，但现有方法仍易受干扰，依赖复杂的多目标损失、指数移动平均、预训练编码器或辅助监督来防止表示坍塌。本文提出LeWorldModel（LeWM），首个可从原始像素端到端稳定训练的JEPA，仅使用两项损失：下一嵌入预测损失与强制潜变量服从高斯分布的正则项。相比唯一现存的端到端方案，可调超参数从六个减少至一个。模型约含1500万可训练参数，可在单张GPU上数小时内完成训练，规划速度比基于基础模型的世界模型快达48倍，且在多种2D和3D控制任务中表现具有竞争力。此外，通过探测物理量，验证了其潜在空间能有效编码有意义的物理结构；意外事件检测实验也表明模型能可靠识别违反物理规律的行为。
原文摘要 · Abstract (English)
Joint Embedding Predictive Architectures (JEPAs) offer a compelling framework for learning world models in compact latent spaces, yet existing methods remain fragile, relying on complex multi-term losses, exponential moving averages, pre-trained encoders, or auxiliary supervision to avoid representation collapse. In this work, we introduce LeWorldModel (LeWM), the first JEPA that trains stably end-to-end from raw pixels using only two loss terms: a next-embedding prediction loss and a regularizer enforcing Gaussian-distributed latent embeddings. This reduces tunable loss hyperparameters from six to one compared to the only existing end-to-end alternative. With ~15M parameters trainable on a single GPU in a few hours, LeWM plans up to 48x faster than foundation-model-based world models while remaining competitive across diverse 2D and 3D control tasks. Beyond control, we show that LeWM's latent space encodes meaningful physical structure through probing of physical quantities. Surprise evaluation confirms that the model reliably detects physically implausible events.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。