arXiv:2605.09241cs.LGcs.AI2026-05被引 6

通过子空间正则化提升世界模型训练稳定性,避免表征崩溃。

Sub-JEPA: Subspace Gaussian Regularization for Stable End-to-End World Models

论文配图:Sub-JEPA: Subspace Gaussian Regularization for Stable End-to-End World Models
图 1 · 摘自论文原文
  • 在多个随机子空间中施加高斯约束,而非原始空间
  • 在4个连续控制环境中性能显著优于LeWM
  • 方法简单有效,适合作为未来研究基线

联合嵌入预测架构(JEPAs)通过预测未来潜在表示来学习世界模型,但其训练存在偏差-方差权衡问题。缺乏足够结构约束时,过度的表示方差会导致模型坍缩至平凡解。近期的LeWorldModel(LeWM)表明,通过施加各向同性高斯先验可缓解该问题。然而,潜在表示本质上位于高维环境空间中的低维流形上,直接在环境空间中施加各向同性高斯先验会引入过强偏差。本文提出Sub-JEPA,通过在多个随机子空间中应用高斯约束,而非原嵌入空间,以放松全局约束,同时保留防坍缩效果,从而在训练稳定性和表征灵活性间取得更好平衡。在四个连续控制环境上的大量实验表明,Sub-JEPA始终以明显优势超越LeWM。该方法简单有效,可作为未来基于JEPA的世界模型研究的强基线。

原文摘要 · Abstract (English)

Joint-Embedding Predictive Architectures (JEPAs) provide a simpleframework for learning world models by predicting future latent representations.However, JEPA training is subject to a bias-variance tradeoff.Without sufficient structural constraints, excessive representationalvariance causes the model to collapse to trivial solutions.The recent LeWorldModel (LeWM) shows that this issue can be alleviated bysimply constraining latent embeddings with an isotropic Gaussian prior.However, latent representations inherently lie on low-dimensional manifoldswithin a high-dimensional ambient space, and enforcing an isotropic Gaussianprior directly in this ambient space introduces an overly strong bias.In this work, we propose ame, which seeks a favorable operatingpoint on the bias-variance frontier by applying Gaussian constraints inmultiple random subspaces rather than in the originalembedding space.This design relaxes the global constraint while preserving itsanti-collapse effect, leading to a better balance between trainingstability and representation flexibility.Extensive experiments across fourcontinuous-control environments demonstrate that consistentlyoutperforms LeWM with very clear margins.Our method is simple yet effective, and serves as a strong baseline for future JEPA-based world model research.fdefinedeeemodeThe code is available at https://github.com/intcomp/Sub-JEPA.

世界模型表征学习正则化强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。