arXiv:2606.19408cs.LGcs.RO2026-06

提出可变长度潜在动作,突破固定瓶颈限制,提升视频动作对齐效果。

FlexLAM: Resolving the Bottleneck Trade-off in Latent Action Learning

论文配图:FlexLAM: Resolving the Bottleneck Trade-off in Latent Action Learning
图 1 · 摘自论文原文
  • 用嵌套丢弃训练可变长度潜在动作,动态调整编码精度。
  • 在不同令牌预算下均超越固定容量模型,低标签场景表现更优。
  • 无需重训练即可按需调整推理令牌数,适合视频理解与世界模型应用。

潜在动作为无动作视频与下游决策之间提供了紧凑接口,但现有潜在动作模型(LAMs)强制所有转换通过固定容量瓶颈。我们识别出瓶颈权衡:过紧的编码会丢弃动作对齐所需转换线索,而过松的编码则保留额外转换变化,在标签稀少或分布狭窄时难以解析。FlexLAM 以可变长度潜在动作替代固定容量,通过嵌套丢弃训练,生成前缀有效编码,先捕捉紧凑转换结构,仅在需要时添加细节,无需新架构或损失函数。单个 FlexLAM 在标准稀疏标签监督下,于各评估令牌预算中表现匹配或超越独立训练的固定容量 LAM;在低回报单任务对齐压力测试中亦表现优异,表明 FlexLAM 不仅推理时可调,且在相同令牌预算下学习到更优潜在动作接口。同一模型支持推理时令牌预算动态调整而无需重训练,且提升了 Ego4D 转换重建效果。结果表明,可变长度潜在动作是潜在动作模型、潜在动作世界模型及视频预训练动作接口中固定容量瓶颈的无架构、即插即用升级方案。

原文摘要 · Abstract (English)

Latent actions provide a compact interface between action-free video and downstream decision-making, yet existing Latent Action Models (LAMs) force every transition through a fixed-capacity bottleneck. We identify a bottleneck trade-off: overly tight codes can discard transition cues needed for action alignment, while overly loose codes preserve additional transition variation that must be resolved when alignment labels are scarce or narrowly distributed. FlexLAM replaces this fixed capacity with variable-length latent actions trained by nested dropout, yielding prefix-valid codes that capture compact transition structure first and add detail only when needed, without new architectures or losses. A single FlexLAM matches or surpasses separately trained fixed-capacity LAMs at every evaluated token budget under standard scarce-label supervision and under a low-return single-task alignment stress test, indicating that FlexLAM is not merely adjustable at inference time but learns a better latent-action interface at the same token budgets. The same model supports inference-time token-budget adjustment without retraining, and FlexLAM improves Ego4D transition reconstruction. These results suggest that variable-length latent actions are an architecture-free, drop-in upgrade to the fixed-capacity bottleneck in latent action models, latent-action world models, and video-pretrained action interfaces.

潜在动作视频理解可变长度模型优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。