针对边缘设备优化贝叶斯推断计算,实现高达5倍加速。
A Hardware-oriented Approach for Efficient Bayesian Inference Computation and Deployment
- 重构张量运算内存布局,提升GPU执行效率。
- 在770个POMDP配置上实现2-5倍加速,结果与基准一致。
- 适用于资源受限场景的高效贝叶斯推断部署,适合嵌入式系统开发者。
贝叶斯推断为不确定性推理提供了严谨框架,但其计算开销限制了在资源受限边缘设备上的部署。本文提出一种面向硬件的离散贝叶斯推断加速方法,针对商用嵌入式GPU优化。我们发现,广泛类别的变分消息传递算法延迟主要由张量收缩决定。通过两种互补的合并策略重构内存布局,生成更紧凑、规则的计算原语,更利于高效GPU执行。进一步引入可选稀疏数组表示和张量聚类方案,降低内存占用。实例化该方法,对隐马尔可夫模型(HMM)的三种消息传递算法(变分滤波、变分消息传递、边际消息传递)生成优化变体。同时设计基于机器学习的自动调优器,可自动选择最优算法变体。在NVIDIA Jetson Orin AGX上对770个随机采样的真实部分可观测马尔可夫决策过程(POMDP)配置进行基准测试,实现最高5倍加速,典型提升2-2.5倍,且输出数值完全一致于基线。
原文摘要 · Abstract (English)
Bayesian inference provides a principled foundation for reasoning under uncertainty, but its computational cost hinders deployment on resource-constrained edge devices. In this paper, we present a hardware-oriented methodology for accelerating discrete Bayesian inference on commercial off-the-shelf embedded GPUs. We identify that the latency of a broad class of variational message-passing algorithms is dominated by tensor contractions. Our approach restructures the memory layout of these operations using two complementary merging strategies that produce compact, regularly-shaped primitives better suited for efficient GPU execution. We then introduce optional sparse array representations and a tensor-clustering scheme to reduce the memory footprint. We instantiate the methodology and produce optimized variants of three message-passing algorithms for Hidden Markov Models (HMMs), namely variational filtering, variational message passing, and marginal message passing. Furthermore, we complement this with a machine-learning-based autotuner that automatically selects the best-performing algorithmic variant for a given generative model specification. Benchmarked on an NVIDIA Jetson Orin AGX across 770 randomly sampled realistic Partially Observable Markov Decision Process (POMDP) configurations, our implementations achieve speedups of up to 5x, with typical gains of 2-2.5x, while producing numerically identical outputs to the baseline implementations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。