让机器人AI模型在不同设备上高效运行的C++通用推理框架
Embodied.cpp: A Portable Inference Runtime of Embodied AI Models on Heterogeneous Robots
- 用分层架构统一处理视觉-语言-动作模型的执行流程
- 实测比Python版快1.05倍到2.7倍，显存占用降低7%-77%
- 适合需要跨机器人部署、低延迟控制的研究者与开发者
具身AI模型涵盖视觉-语言-动作（VLA）模型和世界-动作模型（WAMs），但实际部署仍受限于特定模型的Python栈、后端假设及机器人侧粘合代码，尤其在异构边缘设备上。现有推理运行时多针对请求-响应服务设计，无法满足具身部署的运行时需求：闭环控制内的多速率执行、异构硬件上的延迟优先单批次推理，以及超越固定标记输入输出的可扩展接口。本文提出Embodied.cpp，一个面向具身模型的便携式C++推理运行时。基于对代表性VLA模型和WAMs的架构分析，Embodied.cpp捕捉共享执行路径并组织为五层：输入适配器、序列构建器、主干执行、头插件和部署适配器。该运行时提供模块化多速率执行、延迟优先融合推理及可扩展操作符与I/O支持，通过单一后端抽象实现跨异构设备、机器人和模拟器的部署。我们在三个VLA和两个WAM模型上评估了Embodied.cpp，采用归一化比较方式对比Python与C++量化配置。总体上，相比Python基线，Embodied.cpp实现了1.05倍至2.70倍的推理加速，显存占用降低7%至77%，且多数配置下成功率接近基线。结果表明，Embodied.cpp在提升部署效率的同时，保持了多样化具身模型架构的高质量控制性能。项目链接：https://github.com/SEU-PAISys/Embodied.cpp
原文摘要 · Abstract (English)
Embodied AI models now span vision-language-action (VLA) models and world-action models (WAMs), but practical deployment remains fragmented across model-specific Python stacks, backend assumptions, and robot-side glue code, especially on heterogeneous edge devices. Existing inference runtimes are designed mainly for request-response serving and therefore do not satisfy the runtime contract of embodied deployment: multi-rate execution inside closed-loop control, latency-first batch-1 inference on heterogeneous hardware, and extensible embodied interfaces beyond fixed token I/O. We present Embodied$.$cpp, a portable C++ inference runtime for embodied models. Based on an architectural analysis of representative VLA models and WAMs, Embodied$.$cpp captures a shared execution path and organizes it into five layers: input adapters, sequence builders, backbone execution, head plugins, and deployment adapters. The runtime provides modular multi-rate execution, latency-first fused inference, and extensible operator and I/O support, enabling deployment across heterogeneous devices, robots, and simulators through one backend abstraction. We evaluate Embodied$.$cpp on three VLA and two WAM models, using normalized comparisons across Python and C++ quantization configurations. Overall, Embodied$.$cpp achieves 1.05x-2.70x inference speedups and 7\%-77\% lower VRAM relative to Python baselines, while maintaining near-baseline success for most configurations. These results show that Embodied$.$cpp improves deployment efficiency while preserving high control quality across diverse embodied model architectures. Project Link: https://github.com/SEU-PAISys/Embodied.cpp
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。