arXiv:2508.11584cs.ROcs.AI2025-08被引 1

多任务机器人视觉加速:共享特征,实时运行,省时省内存。

Visual Perception Engine: Fast and Flexible Multi-Head Inference for Robotic Vision Tasks

  • 共享骨干网络特征,多头并行处理任务,避免重复计算。
  • 相比串行执行最高提速3倍,支持50赫兹以上实时推理。
  • 适合资源受限的机器人平台,开源且兼容ROS2生态。

在资源受限的机器人平台上部署多个机器学习模型以完成不同感知任务时,常面临计算冗余、内存占用大和集成复杂等问题。为此,本文提出视觉感知引擎(VPEngine),一个模块化框架,旨在高效利用GPU实现视觉多任务处理,同时保持可扩展性和开发友好性。该框架采用共享基础模型骨干网络提取图像表征,并在多个专用任务头(如深度估计、目标检测、语义分割)间无冗余地共享这些表征,全程避免不必要的GPU-CPU内存传输。这一设计消除了传统串行模型中特征提取的计算冗余,同时支持根据应用需求动态调整任务优先级。我们以DINOv2为基础模型,实现多任务并行推理,在NVIDIA Jetson Orin AGX上使用TensorRT优化模型,达到≥50 Hz的端到端实时性能。基于CUDA Multi-Process Service(MPS),VPEngine实现了高效GPU利用率与恒定内存开销,支持运行时动态调节各任务推理频率。框架以Python编写,提供ROS2 C++(Humble)绑定,便于机器人社区在多种平台中快速集成与使用。

原文摘要 · Abstract (English)

Deploying multiple machine learning models on resource-constrained robotic platforms for different perception tasks often results in redundant computations, large memory footprints, and complex integration challenges. In response, this work presents Visual Perception Engine (VPEngine), a modular framework designed to enable efficient GPU usage for visual multitasking while maintaining extensibility and developer accessibility. Our framework architecture leverages a shared foundation model backbone that extracts image representations, which are efficiently shared, without any unnecessary GPU-CPU memory transfers, across multiple specialized task-specific model heads running in parallel. This design eliminates the computational redundancy inherent in feature extraction component when deploying traditional sequential models while enabling dynamic task prioritization based on application demands. We demonstrate our framework's capabilities through an example implementation using DINOv2 as the foundation model with multiple task (depth, object detection and semantic segmentation) heads, achieving up to 3x speedup compared to sequential execution. Building on CUDA Multi-Process Service (MPS), VPEngine offers efficient GPU utilization and maintains a constant memory footprint while allowing per-task inference frequencies to be adjusted dynamically during runtime. The framework is written in Python and is open source with ROS2 C++ (Humble) bindings for ease of use by the robotics community across diverse robotic platforms. Our example implementation demonstrates end-to-end real-time performance at $\geq$50 Hz on NVIDIA Jetson Orin AGX for TensorRT optimized models.

机器人视觉多任务推理GPU优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。