为自动驾驶设计的高光谱图像分割FPGA系统,实现低延迟高效推理。
Optimization of DNN-based HSI Segmentation FPGA-based SoC for ADS: A Practical Approach
- 分层软硬件协同设计,优化任务分配与数据流
- 模型压缩后仅保留1.02%参数,计算量降至24.34%
- 实现在FPGA上2.86倍加速,精度损失可忽略
高光谱成像(HSI)在自主导航中具有提升检测、跟踪和场景理解系统精度与鲁棒性的潜力。结合深度神经网络(DNN)与小型快照式HSI相机可增强系统可靠性。相比灰度和RGB成像,HSI能更准确反映目标的光谱反射特性与颜色匹配性。尽管基于HSI的视觉系统进展显著,但自动驾驶系统(ADS)对延迟、资源消耗和安全性有严格要求,推动机器学习负载向边缘平台迁移。这需要全面的软硬件协同设计,在有限资源下高效分配与优化任务。尤其针对推理阶段,过度参数化的DNN带来巨大计算挑战;同时,HSI数据预处理复杂且常被忽视,需精细管理内存布局与任务间通信,以实现SoC上的高效集成流水线。本文提出一套面向基于FPGA的SoC部署的DNN-HSI分割处理器实用优化方案,涵盖关键技术:软件/硬件功能分工、硬件感知预处理、模型压缩与完整流水线部署。所用压缩技术将模型计算量降至原规模的24.34%,参数量降至1.02%,推理速度提升2.86倍,且分割精度无明显下降。
原文摘要 · Abstract (English)
The use of HSI for autonomous navigation is a promising research field aimed at improving the accuracy and robustness of detection, tracking, and scene understanding systems based on vision sensors. Combining advanced computer algorithms, such as DNNs, with small-size snapshot HSI cameras enhances the reliability of these systems. HSI overcomes intrinsic limitations of greyscale and RGB imaging in depicting physical properties of targets, particularly regarding spectral reflectance and metamerism. Despite promising results in HSI-based vision developments, safety-critical systems like ADS demand strict constraints on latency, resource consumption, and security, motivating the shift of ML workloads to edge platforms. This involves a thorough software/hardware co-design scheme to distribute and optimize the tasks efficiently among the limited resources of computing platforms. With respect to inference, the over-parameterized nature of DNNs poses significant computational challenges for real-time on-the-edge deployment. In addition, the intensive data preprocessing required by HSI, which is frequently overlooked, must be carefully managed in terms of memory arrangement and inter-task communication to enable an efficient integrated pipeline design on a SoC. This work presents a set of optimization techniques for the practical co-design of a DNN-based HSI segmentation processor deployed on a FPGA-based SoC targeted at ADS, including key optimizations such as functional software/hardware task distribution, hardware-aware preprocessing, ML model compression, and a complete pipelined deployment. Applied compression techniques significantly reduce the complexity of the designed DNN to 24.34% of the original operations and to 1.02% of the original number of parameters, achieving a 2.86x speed-up in the inference task without noticeable degradation of the segmentation accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。