针对自动驾驶视觉模型在异构边缘GPU上的部署难题,提出高效调度与适配方案。
Edge Physical AI Deployment of Vision Transformers on Heterogeneous Edge GPU Targeting Autonomous Vehicles

- 设计帧调度策略,跨GPU与双DLA核心按比例分配任务
- 模型精度仅降2%(F1达92%),实现125.93 FPS、4.0 FPS/W能效
- 适合需低延迟高能效的车载视觉系统开发者
自动驾驶等物理AI系统需要满足严格边缘延迟与能耗约束的Transformer感知模型。然而,异构边缘GPU部署受限于硬件引擎利用率低和加速器不兼容算子,导致执行碎片化与每瓦吞吐量下降。本文提出面向NVIDIA边缘GPU的异构帧调度方法H-FraDS,通过固定调度比例将帧分发至GPU与双深度学习加速器(DLA)核心,提升资源利用率。为支持调度,对不兼容的Transformer组件进行适配:重塑张量、用tanh近似误差函数(ERF)、以有界tanh替换层归一化。适配后模型保持92% F1分数,较原始模型仅降低2%。进一步引入光流加速器(OFA)实现推理端光流估计。据作者所知,此前工作未同时解决上述问题。基于Swin Transformer的自动驾驶感知任务中,H-FraDS平衡调度(1:2)实现125.93 FPS,相较仅使用适配后DLA提升2.36倍,能效达4.0 FPS/W,DLA延迟约24 ms,满足30 FPS实时性要求;GPU-DLA-OFA配置下,DLA吞吐速度提升2.02倍。
原文摘要 · Abstract (English)
Physical AI systems, such as autonomous vehicles and intelligent machines, require transformer-based perception models that satisfy stringent edge latency and energy constraints. However, heterogeneous edge-GPU deployment remains limited by underutilized hardware engines and accelerator-incompatible operators, causing fragmented execution and lower throughput per watt. This paper presents Heterogeneous Frame Dispatch Scheduling (H-FraDS), a hardware-aware frame scheduling methodology for transformer inference on a recent NVIDIA edge GPU. H-FraDS routes frames across the GPU and dual deep learning accelerator (DLA) cores using fixed dispatch ratios to improve utilization under latency and power constraints. To enable scheduling, incompatible transformer components are adapted for DLA execution by reshaping tensors, approximating error function (ERF) with tanh, and replacing layer normalization with bounded tanh. The adapted model maintains a 92% F1 score, with only a 2% reduction from the original. Optical flow accelerator (OFA) is further used for inference-side optical-flow estimation. To the best of the authors' knowledge, prior work has not addressed these combined issues. Using Swin Transformer for autonomous-driving perception, H-FraDS Balanced Dispatch (1:2) achieves 125.93 FPS, a 2.36x speedup over standalone adapted-DLA execution, 4.0 FPS/W, and approximately 24 ms DLA latency, satisfying 30 FPS real-time operation; the GPU-DLA-OFA case achieves a 2.02x DLA throughput speedup.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。