无需搜索即可直接生成动作,让世界模型更高效实用。
INTACT: Isomorphic Intent-to-Action Learning for Search-Free World Models

- 用意图对齐架构实现从视觉到动作的端到端映射。
- 单轮训练零搜索达97%以上成功率,采样量减少23倍。
- 适合需要快速决策的机器人控制与强化学习场景。
前向隐空间世界模型可预测动作对场景的影响,但为达成目标动作需依赖昂贵的推理时搜索。本文提出INTACT(INtent-To-ACTion),一种端到端的JEPA框架,将无奖励、带动作标签的轨迹转化为可部署的意图-动作接口。每个状态转移提供物理意图差 $z_{t+1}-z_t$,未来目标则提供部署意图差 $ ext{sg}(z_g)-z_t$。通过相同的四槽语法和共享参数,使局部与目标意图骨干网络在结构上同构,并基于同一预测器的行动律语义建立意图家族间的对应关系,而非点对点隐空间相等。该方法实现了从RGB观测到有效动作意图坐标的完整迁移,以及意图家族到对应行动律家族的跨域迁移。非对称端点梯度锚定物理后继并固定未来目标,将表示学习与控制统一,无需点对点隐空间匹配或全局线性动力学假设。所得坐标支持鲁棒的分布式动作律:其条件均值可直接作为无搜索策略,采样仍可用于多样性或可选验证。在四个官方LeWM任务上,单轮训练、零搜索模型分别达到85.78%、100.00%、97.67%、97.89%成功;以直接规划为中心的局部CEM仅需384个候选序列(原9,000)即达96.86%宏观成功率,采样降低23.44倍,且优于纯CEM 16.00个百分点。一个共享的四任务编码器取得89.39% E5 Direct宏平均,且每项任务均超越联合训练的LeWM;预测动作族与专家动作族的kNN相关性达 $r=0.954$。直接推理耗时仅2.9–5.5毫秒。
原文摘要 · Abstract (English)
Forward latent world models predict how actions change a scene, but recover actions for a desired change only through expensive test-time search. We introduce INTACT (INtent-To-ACTion), an end-to-end JEPA that turns action-labeled, reward-free trajectories into a deployable intent-to-action interface. Each transition supplies physical intent $z_{t+1}-z_t$, while a future goal supplies deployment intent $\operatorname{sg}(z_g)-z_t$. The architecture is isomorphic between the local and goal motion-intent backbone-input graphs through an identical four-slot grammar and shared parameters, and between supported local and goal motion-intent families through action-law semantics induced by the same predictor rather than pointwise latent equality. INTACT also provides intact transfer from RGB evidence to action-effective latent intent coordinates and from intent families to their corresponding action-law families. Asymmetric endpoint gradients ground physical successors and fix future goals as anchors, joining representation learning and control without pointwise latent matching or globally linear dynamics. The resulting coordinates support a robust distributional action law: its conditional mean serves directly as a search-free policy, while sampling remains available for diversity or optional verification. On the four official LeWM tasks, one-epoch, zero-search models reach 85.78\%, 100.00\%, 97.67\%, and 97.89\% success. Optional local CEM centered on the Direct plan reaches 96.86\% macro success using 384 instead of 9,000 candidate sequences, reducing sampling by $23.44\times$ while improving pure CEM by 16.00 points. One shared four-task encoder reaches 89.39\% E5 Direct macro and improves every task over jointly trained LeWM, while predicted--expert action-family kNN tracks Direct success at $r=0.954$. Direct inference takes 2.9--5.5 ms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。