用超网络只激活小部分参数，让机器人视觉语言模型推理快90倍
HyperVLA: Efficient Inference in Vision-Language-Action Models via Hypernetworks
- 用超网络动态生成任务专用策略，推理时只激活少量参数
- 零样本泛化成功率与顶尖模型相当，但测试参数量减少90倍
- 适合部署在资源受限设备的通用机器人控制场景
基于具备强泛化能力的语言与视觉基础模型，并在大规模机器人数据上训练的视觉-语言-动作（VLA）模型，已成为学习通用机器人策略的有前景方法。然而，现有VLA模型存在推理成本极高的关键问题。本文提出HyperVLA以解决该问题。不同于传统单体VLA在训练和推理中均激活整个模型，HyperVLA采用新型超网络（HN）架构，推理时仅激活一小部分任务专用策略，同时在训练中仍保留高模型容量以支持多样化的多任务行为。成功训练基于超网络的VLA面临挑战，因此HyperVLA包含若干关键算法设计：合理利用现有视觉基础模型的先验知识、超网络归一化及一种动作生成策略。相比单体VLA，HyperVLA在零样本泛化和少样本适应上的成功率相似甚至更高，同时显著降低推理成本。相较于当前最先进的VLA模型OpenVLA，HyperVLA在测试时将激活参数数量减少90倍，推理速度提升120倍。代码已公开于https://github.com/MasterXiong/HyperVLA。
原文摘要 · Abstract (English)
Built upon language and vision foundation models with strong generalization ability and trained on large-scale robotic data, Vision-Language-Action (VLA) models have recently emerged as a promising approach to learning generalist robotic policies. However, a key drawback of existing VLAs is their extremely high inference costs. In this paper, we propose HyperVLA to address this problem. Unlike existing monolithic VLAs that activate the whole model during both training and inference, HyperVLA uses a novel hypernetwork (HN)-based architecture that activates only a small task-specific policy during inference, while still retaining the high model capacity needed to accommodate diverse multi-task behaviors during training. Successfully training an HN-based VLA is nontrivial so HyperVLA contains several key algorithm design features that improve its performance, including properly utilizing the prior knowledge from existing vision foundation models, HN normalization, and an action generation strategy. Compared to monolithic VLAs, HyperVLA achieves a similar or even higher success rate for both zero-shot generalization and few-shot adaptation, while significantly reducing inference costs. Compared to OpenVLA, a state-of-the-art VLA model, HyperVLA reduces the number of activated parameters at test time by $90\times$, and accelerates inference speed by $120\times$. Code is publicly available at https://github.com/MasterXiong/HyperVLA
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。