模型自监督提炼动作预测的具身推理,无需人工标注。
Self-Supervised Bootstrapping of Action-Predictive Embodied Reasoning
- 用变分推断将推理设为隐变量,自动筛选关键动作信号。
- 模拟与真实机器人上实现28%操控成功率提升,导航得分翻倍。
- 适合构建无需人工设计的自主智能体,尤其在复杂物理环境。
具身思维链(CoT)推理显著提升了视觉-语言-动作(VLA)模型性能,但现有方法依赖固定模板定义推理单元(如场景物体、高层计划、结构可操作性),导致策略处理无关信息,干扰关键动作预测信号。这形成瓶颈:无成功策略则无法验证推理质量;无高质量推理则难以构建鲁棒策略。本文提出R&B-EnCoRe,通过自监督精炼,从互联网级知识中自举具身推理能力。将推理视为重要性加权变分推断中的隐变量,模型可生成并蒸馏出面向具体执行的优化推理数据集,无需外部奖励、验证器或人工标注。我们在多种具身形态上验证该方法:模拟中的Franka Panda、硬件上的WidowX机械臂,双足、轮式、自行车、四足步行机器人,以及自动驾驶任务。使用1B、4B、7B、30B参数量的多种VLA架构进行测试。结果表明,相比盲目推理所有可用单元的模型,本方法在操控任务上提升28%成功率,在导航评分上提高101%,碰撞率降低21%。R&B-EnCoRe使模型能提炼出对成功控制有预测性的推理策略,既避免了人工标注工程,又将互联网级知识落地于物理执行。
原文摘要 · Abstract (English)
Embodied Chain-of-Thought (CoT) reasoning has significantly enhanced Vision-Language-Action (VLA) models, yet current methods rely on rigid templates to specify reasoning primitives (e.g., objects in the scene, high-level plans, structural affordances). These templates can force policies to process irrelevant information that distracts from critical action-prediction signals. This creates a bottleneck: without successful policies, we cannot verify reasoning quality; without quality reasoning, we cannot build robust policies. We introduce R&B-EnCoRe, which enables models to bootstrap embodied reasoning from internet-scale knowledge through self-supervised refinement. By treating reasoning as a latent variable within importance-weighted variational inference, models can generate and distill a refined reasoning training dataset of embodiment-specific strategies without external rewards, verifiers, or human annotation. We validate R&B-EnCoRe across manipulation (Franka Panda in simulation, WidowX in hardware), legged navigation (bipedal, wheeled, bicycle, quadruped), and autonomous driving embodiments using various VLA architectures with 1B, 4B, 7B, and 30B parameters. Our approach achieves 28% gains in manipulation success, 101% improvement in navigation scores, and 21% reduction in collision-rate metric over models that indiscriminately reason about all available primitives. R&B-EnCoRe enables models to distill reasoning that is predictive of successful control, bypassing manual annotation engineering while grounding internet-scale knowledge in physical execution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。