让零值更有信息量,用二值化U-Net实现边缘设备实时图像分割。
Zeros can be Informative: Masked Binary U-Net for Image Segmentation on Tensor Cores
- 通过代价感知掩码策略,智能选择二值化位置以平衡精度与效率。
- 在3个基准上比16位浮点U-Net快2.04倍,能耗降低3.54倍,精度仅降3%。
- 专为GPU张量核心设计,支持高效二值计算,适合部署于边缘设备。
实时图像分割是AR/VR、机器人、无人机和自动驾驶系统的关键技术,需在资源受限的边缘设备上满足精度、延迟和能耗的严格约束。尽管U-Net相比大型Transformer模型在精度与效率间取得较好平衡,但在高分辨率输入下实现实时性能仍受计算、内存和功耗限制。极端量化,特别是二值网络,因其硬件友好性备受关注。然而,两大障碍限制其实用性:(1) 精度严重下降;(2) 缺乏可在通用GPU上实现端到端高效性的方案。我们提出两个实证观察:(1) 显式零状态至关重要:训练时对二值化U-Net权重施加零掩码可带来显著稀疏性;(2) 各层量化敏感度均匀。基于此,我们提出掩码二值化U-Net(MBU-Net),通过成本感知掩码策略,优先在单位代价收益最高的位置进行掩码,实现近二值效率与精度的统一。为落地应用,我们开发了基于减法比特编码方案的GPU执行框架,将MBU-Net映射至张量核心,高效实现带掩码的二值权重与二值激活。该设计利用原生二值张量核心BMMA指令,显著提升吞吐并降低能耗。在3个分割基准上,MBU-Net达到接近全精度的性能(平均精度下降3%),相较16位浮点U-Net实现2.04倍加速和3.54倍能效提升。
原文摘要 · Abstract (English)
Real-time image segmentation is a key enabler for AR/VR, robotics, drones, and autonomous systems, where tight accuracy, latency, and energy budgets must be met on resource-constrained edge devices. While U-Net offers a favorable balance of accuracy and efficiency compared to large transformer-based models, achieving real-time performance on high-resolution input remains challenging due to compute, memory, and power limits. Extreme quantization, particularly binary networks, is appealing for its hardware-friendly operations. However, two obstacles limit practicality: (1) severe accuracy degradation, and (2) a lack of end-to-end implementations that deliver efficiency on general-purpose GPUs. We make two empirical observations that guide our design. (1) An explicit zero state is essential: training with zero masking to binary U-Net weights yields noticeable sparsity. (2) Quantization sensitivity is uniform across layers. Motivated by these findings, we introduce Masked Binary U-Net (MBU-Net), obtained through a cost-aware masking strategy that prioritizes masking where it yields the highest accuracy-per-cost, reconciling accuracy with near-binary efficiency. To realize these gains in practice, we develop a GPU execution framework that maps MBU-Net to Tensor Cores via a subtractive bit-encoding scheme, efficiently implementing masked binary weights with binary activations. This design leverages native binary Tensor Core BMMA instructions, enabling high throughput and energy savings on widely available GPUs. Across 3 segmentation benchmarks, MBU-Net attains near full-precision accuracy (3% average drop) while delivering 2.04x speedup and 3.54x energy reductions over a 16-bit floating point U-Net.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。