用序列顺序匹配提升单段错时视频的模仿学习效果
Imitation Learning from a Single Temporally Misaligned Video
- 以序列顺序覆盖为标准，而非逐帧对齐
- 在错时演示下，平均收益提升4.5倍至6.6倍
- 适合处理动作时序不一致的真实示范场景
我们研究从单段视觉示范中学习序列任务的问题。当示范因时间差异、身体形态不同或执行不一致导致时序错位时，现有方法通过帧级对齐实现分布匹配，但无法保证时间顺序或一致进展。我们的核心洞察是：匹配应基于序列层面。理想匹配发生在一方序列按正确顺序覆盖另一方的所有子目标时。为此提出ORCA（有序覆盖对齐），一种密集的每步奖励函数，衡量代理以正确顺序覆盖示范帧的概率。在时序错位的示范下，使用ORCA训练的智能体在Meta-world任务上平均归一化回报从0.11提升至0.50（4.5倍提升），在Humanoid-v4任务上从6.55提升至43.3（6.6倍提升），优于最优帧级匹配算法。实证分析还表明ORCA对不同水平的时序错位具有鲁棒性。代码已开源。
原文摘要 · Abstract (English)
We examine the problem of learning sequential tasks from a single visual demonstration. A key challenge arises when demonstrations are temporally misaligned due to variations in timing, differences in embodiment, or inconsistencies in execution. Existing approaches treat imitation as a distribution-matching problem, aligning individual frames between the agent and the demonstration. However, we show that such frame-level matching fails to enforce temporal ordering or ensure consistent progress. Our key insight is that matching should instead be defined at the level of sequences. We propose that perfect matching occurs when one sequence successfully covers all the subgoals in the same order as the other sequence. We present ORCA (ORdered Coverage Alignment), a dense per-timestep reward function that measures the probability of the agent covering demonstration frames in the correct order. On temporally misaligned demonstrations, we show that agents trained with the ORCA reward achieve $4.5$x improvement ($0.11 \rightarrow 0.50$ average normalized returns) for Meta-world tasks and $6.6$x improvement ($6.55 \rightarrow 43.3$ average returns) for Humanoid-v4 tasks compared to the best frame-level matching algorithms. We also provide empirical analysis showing that ORCA is robust to varying levels of temporal misalignment. Our code is available at https://github.com/portal-cornell/orca/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。