让视觉语言动作模型在不微调的情况下自适应复杂环境。
RoboHarness: A Memory-Augmented Policy Harness for Vision-Language-Action Model Robustness via In-Context Adaptation
- 用双记忆检索增强生成实现在线动态调整策略
- 在长任务链上成功率提升89.1%，平均提升56.6%
- 适合希望提升机器人泛化能力的研究者和开发者
尽管视觉-语言-动作（VLA）模型作为通用机器人控制器前景广阔，但其在分布外（OOD）任务中对感知噪声和环境变化的鲁棒性仍受限于缺乏长期记忆、因果故障归因与动态干预能力。为此，我们提出RoboHarness——一种无需参数微调即可实现稳健上下文自适应的记忆增强策略框架。该框架通过在线对比双记忆检索增强生成（RAG）、基于归因驱动的多模态大模型视觉语言协调器以及可扩展的模型上下文协议（MCP）干预机制运行，同时离线的内存整合模块持续将执行轨迹提炼为可靠先验。在LIBERO-PRO及我们提出的LIBERO-RoboHarness基准上，对三种主干模型（π₀, π₀.₅, SmolVLA）的实验表明，RoboHarness实现了平均56.6%的绝对成功率提升，其中长时序任务链提升达89.1%。项目主页与源码见https://github.com/LZY-1021/RoboHarness。
原文摘要 · Abstract (English)
Despite the promise of Vision-Language-Action (VLA) models as generalist robotic controllers, their robustness against perceptual noise and environmental variations in out-of-distribution (OOD) tasks remains fundamentally limited by the absence of long-term memory, causal failure attribution, and dynamic intervention capability. To address this, we propose RoboHarness, a memory-augmented policy harness that upgrades frozen VLA policies for robust in-context adaptation without parameter fine-tuning. Specifically, RoboHarness operates through an online pipeline of contrastive Dual-Memory Retrieval-Augmented Generation (RAG), an attribution-driven vision-language orchestrator implemented with a multimodal large language model, and extensible Model Context Protocol (MCP) interventions, while an offline Memory Consolidation module continuously distills the execution traces into reliable priors. Experimental evaluations across three backbone models ($π_0$, $π_{0.5}$, and SmolVLA) on LIBERO-PRO and our proposed LIBERO-RoboHarness benchmarks demonstrate that RoboHarness achieves an average absolute success rate gain of 56.6%. This includes a significant absolute improvement of 89.1% in long-horizon task chaining. The project page and source code are available at https://github.com/LZY-1021/RoboHarness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。