将通用视觉语言模型适配到人形机器人全身协同操作,提升真实场景下的任务表现。
HAF: Adapting Generalist VLAs to Humanoid Whole-Body Loco-manipulation via Hierarchical Action Flow and Spectral Latent RL

- 分阶段生成动作流,保留身体运动的连贯性与依赖关系
- 在低维噪声空间中优化策略,避免直接微调大模型
- 适合需要高效真实世界部署的复杂人形机器人任务
人形机器人在以人为中心的环境中具有广泛应用前景,但通用视觉-语言-动作(VLA)基础模型难以直接应用于其全身协同运动控制。由于人形机器人动作维度高且各部分高度耦合,传统单阶段VLA架构难以有效协调行走、躯干姿态和双臂操作。此外,离线行为克隆训练的策略在真实部署中仍可能表现不佳。虽然在线强化学习可通过真实交互优化策略,但直接微调大型VLA主干网络计算成本过高,且存在安全风险。为此,本文提出HAF(人形适应框架),由两部分组成:HAF-VLA通过预训练的流匹配VLA构建分层动作流生成器,将全身动作去噪分解为三个有序阶段,利用阶段嵌入与跨阶段键值缓存保留运动学依赖,避免一次性生成导致的动作不协调。在此冻结的HAF-VLA基础上,HAF-Steer采用基于流匹配可逆性和DCT的降维方法,将强化学习优化限制在紧凑的噪声子空间中,训练正则化的SAC策略。该设计避免更新大模型主干,实现高效真实世界策略优化。在七个真实人形机器人协同操作任务上,HAF优于基线单阶段VLA模型,显著提升全身协调性与任务完成率。
原文摘要 · Abstract (English)
Humanoid robots hold great promise as general-purpose agents in human-centered environments, yet generalist vision-language-action (VLA) foundation models are not readily applicable to humanoid whole-body loco-manipulation. The high dimensionality and interdependence of humanoid motions make it challenging for conventional single-stage VLA architectures to coordinate locomotion, waist posture, and dual-arm manipulation effectively. Moreover, policies trained through offline behavior cloning can remain suboptimal during real-world deployment. Although online reinforcement learning can refine policies through real-world interaction, directly tuning large VLA backbones demands excessive computation and may introduce safety risks during real-robot exploration. To address these bottlenecks, we introduce HAF (Humanoid Adaptation Framework), a two-part framework consisting of HAF-VLA and HAF-Steer that transfers off-the-shelf generalist VLA foundation models to humanoid whole-body loco-manipulation. HAF-VLA is a hierarchical action-flow generator built on a pretrained flow-matching VLA. It splits full-body action denoising into three sequential stages with stage embeddings and cross-stage KV caches that retain kinematic dependencies, avoiding incoherent whole-body actions from one-shot generation. On top of the frozen HAF-VLA, HAF-Steer is a latent offline-to-online RL pipeline that leverages flow-matching invertibility and DCT-based dimensionality reduction to restrict RL optimization to a compact noise subspace and train a regularized SAC policy. This avoids updating the large VLA backbone and enables efficient real-world policy refinement. Evaluated on seven real-world humanoid loco-manipulation tasks, HAF surpasses vanilla single-stage VLA baselines and improves whole-body coordination and task performance. Project website: https://grange007.github.io/HAF .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。