用关键帧记忆提升机器人长任务执行成功率
KEMO: Event-Driven Keyframe Memory for Long-Horizon Robot Manipulation with VLA Policies

- 基于事件触发自动挑选关键帧,减少冗余信息
- 实测任务成功率提升23.6%,阶段完成率提升34.1%
- 适合需要长期规划的多步机器人操作任务
长时序机器人操作仍具挑战,因相似观测可能出现在不同执行阶段,而动作选择依赖于已完成的操作历史。记忆机制可帮助策略通过执行历史推断任务进展。然而,现有方法或保留密集历史需压缩,或仅依赖近期上下文而忽略早期关键事件。本文提出KEMO,一种轻量级插件式记忆框架,为VLA策略自动筛选与任务相关状态变化的关键帧。KEMO结合机器人运动学与视觉过滤检测事件,将选定关键帧编码为紧凑的时间有序记忆令牌,并通过交叉注意力与门控残差融合整合至当前视觉特征中用于训练。检测到的事件还用于定义关键过渡附近的高权重训练样本。我们在涵盖2至6个评分子任务、轨迹长度830至2846步(持续时间28至95秒)的真实双臂操作任务上评估KEMO。相比无记忆基线(如$π_{0.5}$),KEMO使综合任务成功率提升23.6%,阶段完成率提升34.1%。消融实验表明,事件驱动的关键帧选择优于均匀采样与近期帧保留;门控融合与关键帧对齐损失加权带来互补增益。
原文摘要 · Abstract (English)
Long-horizon robot manipulation remains challenging because similar observations may occur at different execution stages, while the appropriate action depends on previously completed operations. Memory can address this ambiguity by enabling policies to infer task progress from execution history. However, existing memory-augmented approaches often either retain dense histories that require compression or rely primarily on recent context that may discard earlier task-relevant events. In this work, we propose propose KEMO, a lightweight plug-in memory framework that automatically selectively preserves keyframes associated with task-relevant state changes for VLA policies. KEMO combines robot kinematics with visual filtering to detect events, encodes the selected keyframes as compact temporally ordered memory tokens, and integrates them with current visual features through cross-attention and gated residual fusion for VLA training. The detected events also define higher-weight training samples near critical transitions. We evaluate KEMO on various real-world dual-arm manipulation tasks spanning 2 to 6 scored subtasks, and trajectory length ranging from 830 steps to 2846 execution steps (durations from 28 to 95 seconds). Compared with the memory-free baseline (e.g., $π_{0.5}$), KEMO improves aggregate Task Success Rate by 23.6\% and Stage Completion Rate by 34.1\%. Ablations show that event-driven keyframe selection outperforms uniform sampling and recent-frame retention, while the proposed gated fusion and keyframe-aligned loss weighting provide complementary gains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。