arXiv:2607.12659cs.ROcs.AI2026-07被引 1

让机器人在低功耗设备上实时运行视觉语言动作模型。

Jetson-PI: Towards Onboard Real-Time Robot Control via Foresight-Aligned Asynchronous Inference

论文配图:Jetson-PI: Towards Onboard Real-Time Robot Control via Foresight-Aligned Asynchronous Inference
图 1 · 摘自论文原文
  • 用未来预测模块纠正异步推理中的感知与执行错位问题。
  • 在Jetson Orin上实现8.66倍的控制频率提升,成功率高14.8%。
  • 适合做边缘部署的机器人实时控制,尤其关注低延迟场景。

视觉-语言-动作(VLA)模型在多种具身任务中表现优异,但因其计算复杂度高,在低功耗机载设备(如Jetson Orin)上的部署仍面临挑战,导致推理延迟大、控制频率低。异步推理虽可部分掩盖延迟,但引入感知-执行错位和长响应时间两大问题。本文提出Jetson-PI,通过前瞻对齐的异步校正实现高效部署。为解决错位问题,训练轻量级未来修正模块,基于已执行动作预测未来环境表征,使动作专家可直接基于未来状态预测动作。为降低反应时间,引入基于置信度的调度优化,自适应平衡视觉语言模型与动作专家调用,并结合系统级加速技术:CUDA图复用、GPU驻留中间缓存、流程展开。大量实验表明,相比原生PyTorch和vla.cpp,Jetson-PI在NVIDIA Jetson Orin上控制频率分别提升8.66倍和5.41倍;在LIBERO基准测试中,平均成功率较VLASH高出14.8%。代码已在https://github.com/PKU-SEC-Lab/Jetson-PI发布,基于llama.cpp的高效推理引擎见https://github.com/PKU-SEC-Lab/Jetson-PI-Edge。

原文摘要 · Abstract (English)

Vision-Language-Action (VLA) models have achieved impressive performance on diverse embodied tasks. However, deploying VLA models on low-power onboard devices, such as the Jetson Orin, remains challenging due to their high computational complexity, which leads to substantial inference latency and low control frequency. Asynchronous inference can partially mask this latency by parallelizing action execution and subsequent inference, but it introduces two critical issues: perception-execution misalignment and long reaction time. In this paper, we propose Jetson-PI, a method for efficient VLA deployment on onboard devices via Foresight-Aligned Asynchronous Correction. To address misalignment, we train a lightweight future correction module that predicts future environment representation conditioned on committed actions, enabling the action expert to directly predict actions from the future time step. To reduce reaction time, we introduce confidence-based scheduling optimization that adaptively balances VLM and action expert invocations, complemented by system-level accelerations including CUDA graph reuse, GPU-resident intermediate buffering, and flow unrolling. Extensive experiments demonstrate that Jetson-PI achieves 8.66x and 5.41x improvements in control frequency compared with naive PyTorch and vla.cpp on NVIDIA Jetson Orin, while outperforming VLASH by 14.8\% in average success rate on the LIBERO benchmark. The code of our asynchronous algorithm is available on https://github.com/PKU-SEC-Lab/Jetson-PI, and our efficient llama.cpp-based inference engine is available on https://github.com/PKU-SEC-Lab/Jetson-PI-Edge.

机器人控制边缘计算异步推理VLA模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。