用轨迹可达性度量提升世界模型控制,让动作选择更准。
World Model Control by Trajectory Reachability Metrics
- 引入轨迹可达性度量TRM,基于历史轨迹训练临时成本函数。
- 在TwoRoom任务中成功率从7.0%提升至96.7%,显著改善动作排序。
- 适用于需要精准动作决策的强化学习场景,尤其关注潜空间信息重加权。
潜在世界模型虽能学习控制所需表征,但下游控制器仅依赖终端潜空间距离时仍会表现不佳。本文研究固定编码器下的这一失效问题,提出轨迹可达性度量(TRM),一种从日志轨迹中训练的小型时序成对代价函数,用于对比候选动作序列预测终点与目标的匹配度。在高距离范围的100个两室环境评估中,原始控制器使用LeWorldModel的平均成功率为7.0%;而排除评估集后训练的时序TRM将成功率提升至96.7%,随机标签控制保持0.0%;同时PLDM性能从32.7%升至84.0%。独立的TopoNav像素导航任务也验证了修复排序与闭环控制间的关联。通过共享候选选择审计(SASC)与行空间干预分析,发现性能提升源于重新加权携带动作决策信息的潜空间方向。其中,XY行空间贡献不足1%的终端潜空间均方误差,却承载了主要控制信息。覆盖与边界测试揭示适用范围:平衡门道覆盖可将0.0%成功率恢复至100.0%;未见过的墙朝向与接触丰富的PushT任务暴露了布局、动力学及恢复能力的极限。
原文摘要 · Abstract (English)
Latent world models can learn representations that contain information needed for control, while the downstream controller may still rank candidate actions poorly when it relies on terminal latent distance alone. We study this failure in a fixed encoder and introduce trajectory reachability metrics (TRM), a small temporal pairwise cost trained from logged trajectories and used to rank predicted endpoints of a candidate action sequence against a goal. In the TwoRoom evaluation on 100 episodes from a high distance range, the original controller with LeWorldModel reaches 7.0% mean success. Temporal TRM trained after excluding all evaluation episodes reaches 96.7%, shuffled label controls stay at 0.0%, and TRM also improves PLDM from 32.7% to 84.0%. TopoNav, a separate pixel navigation task, shows the same link between repaired ranking and closed loop control. The selection audit with shared candidates (SASC) and rowspace interventions show that the gain comes from reweighting latent directions carrying the action decision. The XY rowspace contributes less than 1% of terminal latent MSE but carries most of the information needed for control. Coverage and boundary tests define the scope. Balanced doorway coverage restores a 0.0% coverage failure to 100.0% success, while an unseen wall orientation and contact rich PushT expose layout, dynamics, and recovery limits.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。