arXiv:2509.11225cs.ROcs.AI2025-09被引 1

MEMBOT用记忆模块让机器人在传感器时断时续时仍能稳定执行任务。

MEMBOT: Memory-Based Robot in Intermittent POMDP

  • 分两阶段训练:先学通用状态编码,再微调具体任务策略。
  • 在50%观测丢失下仍保持80%峰值性能,优于无记忆和简单循环模型。
  • 适合需要抗干扰、可迁移的现实机器人控制场景。

实际环境中机器人常面临部分甚至间歇性观测缺失问题,如传感器噪声、遮挡或故障。传统强化学习假设可观测全部状态,难以应对此类挑战。本文提出MEMBOT,一种基于记忆的模块化架构,用于解决机器人控制中的间歇性部分可观测性问题。MEMBOT通过两阶段训练:首先离线进行多任务预训练,利用重构损失学习一个任务无关的潜在状态编码器;随后使用行为克隆对特定任务策略进行微调。该编码器采用状态空间模型(SSM)与LSTM结构,整合观测与动作的时间序列,生成即使在观测丢失时也能持续存在的隐状态表示。我们在MetaWorld和Robomimic的10个机器人操作基准任务上评估MEMBOT,测试不同观测丢弃率下的表现。结果表明,MEMBOT始终优于无记忆和简单循环基线,在50%观测可用性下仍保持高达80%的峰值性能。这些发现验证了显式信念建模在实现鲁棒、可迁移、数据高效机器人策略方面的有效性。

原文摘要 · Abstract (English)

Robotic systems deployed in real-world environments often operate under conditions of partial and often intermittent observability, where sensor inputs may be noisy, occluded, or entirely unavailable due to failures or environmental constraints. Traditional reinforcement learning (RL) approaches that assume full state observability are ill-equipped for such challenges. In this work, we introduce MEMBOT, a modular memory-based architecture designed to address intermittent partial observability in robotic control tasks. MEMBOT decouples belief inference from policy learning through a two-phase training process: an offline multi-task learning pretraining stage that learns a robust task-agnostic latent belief encoder using a reconstruction losses, followed by fine-tuning of task-specific policies using behavior cloning. The belief encoder, implemented as a state-space model (SSM) and a LSTM, integrates temporal sequences of observations and actions to infer latent state representations that persist even when observations are dropped. We train and evaluate MEMBOT on 10 robotic manipulation benchmark tasks from MetaWorld and Robomimic under varying rates of observation dropout. Results show that MEMBOT consistently outperforms both memoryless and naively recurrent baselines, maintaining up to 80% of peak performance under 50% observation availability. These findings highlight the effectiveness of explicit belief modeling in achieving robust, transferable, and data-efficient policies for real-world partially observable robotic systems.

机器人控制部分可观测记忆机制强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。