让机器人通过自适应记忆重构，用10秒历史精准识别操作阶段。
Resolving State Ambiguity in Robot Manipulation via Adaptive Working Memory Recoding
- 设计分层特征提取与范围感知路由，实现多时长历史压缩。
- 支持300帧（约10秒）历史窗口，推理速度超20Hz。
- 适合复杂操作中状态模糊的机器人任务，如装配、分拣。
机器人操作中状态模糊普遍存在：相同观测可能对应多个有效行为轨迹。视觉运动策略需从历史中准确提取合适类型和层次的信息以判断当前任务阶段。然而，简单延长历史窗口计算开销大且易过拟合。受人类持续推理与工作记忆重构启发，本文提出PAM——一种具备自适应工作记忆的新型视觉运动策略。采用两阶段训练，仅增加少量成本，即可支持300帧（约10秒）历史窗口，同时保持高于20Hz的推理速度。具体而言，分层帧特征提取器生成运动基元与时间消歧两种表征；范围特异性查询的上下文路由模块，实现跨多时长历史的紧凑特征表示；引入历史信息重建辅助目标，确保路由模块作为有效瓶颈。在7个精心设计的任务上验证，PAM可同时处理多种状态模糊场景。
原文摘要 · Abstract (English)
State ambiguity is common in robotic manipulation. Identical observations may correspond to multiple valid behavior trajectories. The visuomotor policy must correctly extract the appropriate types and levels of information from the history to identify the current task phase. However, naively extending the history window is computationally expensive and may cause severe overfitting. Inspired by the continuous nature of human reasoning and the recoding of working memory, we introduce PAM, a novel visuomotor Policy equipped with Adaptive working Memory. With minimal additional training cost in a two-stage manner, PAM supports a 300-frame history window while maintaining high inference speed. Specifically, a hierarchical frame feature extractor yields two distinct representations for motion primitives and temporal disambiguation. For compact representation, a context router with range-specific queries is employed to produce compact context features across multiple history lengths. And an auxiliary objective of reconstructing historical information is introduced to ensure that the context router acts as an effective bottleneck. We meticulously design 7 tasks and verify that PAM can handle multiple scenarios of state ambiguity simultaneously. With a history window of approximately 10 seconds, PAM still supports stable training and maintains inference speeds above 20Hz. Project website: https://tinda24.github.io/pam/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。