用流匹配建模未来潜空间轨迹，提升视觉噪声下的世界模型鲁棒性。
Flow-JEPA: Flow Matching for Robust Latent Dynamics in JEPA World Models
- 用条件流匹配替代逐步预测，联合生成多步未来潜状态。
- 干净条件下成功率从86%提至92%，噪声下从67%提至86%。
- 适合做高鲁棒性视觉预测的科研与工程人员参考。
联合嵌入预测架构（JEPAs）在学习紧凑预测表示方面展现出巨大潜力，LeWorldModel（LeWM）将这一范式扩展到无需重建的像素级潜空间世界建模。然而，其确定性自回归预测器通过重复单步转移生成未来状态，易积累误差且对任务无关的视觉扰动敏感。本文提出Flow-JEPA（F-JEPA），一种条件流匹配动力学模型，可联合生成以当前观测和动作条件的未来潜状态序列。以高斯分布为流源，模型学习将受扰潜轨迹向清洁未来表示传输，从而暴露于扰动路径并优化向量场。该方法保持无重建的JEPA框架，但将点式转移回归替换为随机轨迹级预测。实验表明，F-JEPA在干净观测下将平均成功率从86%提升至92%，在噪声条件下从67%提升至86%，证明条件流匹配是JEPA世界模型中确定性自回归动力学的有力替代方案。
原文摘要 · Abstract (English)
Joint-Embedding Predictive Architectures (JEPAs) have shown strong potential for learning compact predictive representations, and LeWorldModel (LeWM) extends this paradigm to reconstruction-free latent world modeling from pixels. However, its deterministic autoregressive predictor generates future states through repeated one-step transitions, which can accumulate errors and remain sensitive to task-irrelevant visual perturbations. In this work, we propose Flow-JEPA (F-JEPA), a conditional flow matching dynamics model that jointly generates a sequence of future latent states conditioned on the current observation and actions. A Gaussian distribution serves as the flow source, exposing the vector field to perturbed latent trajectories as it learns to transport them toward clean future representations. This formulation retains the reconstruction-free JEPA framework while replacing point-wise transition regression with stochastic trajectory-level prediction. F-JEPA raises mean success from $86\%$ to $92\%$ under clean observations and from $67\%$ to $86\%$ under noisy conditions, suggesting that conditional flow matching provides a promising alternative to deterministic autoregressive dynamics in JEPA world models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。