用分段线性拟合设计可重构激活单元，大幅降低硬件开销。
GRAU: Generic Reconfigurable Activation Unit Design for Neural Network Hardware Accelerators
- 基于2的幂次斜率的分段线性拟合，仅需比较器和1位右移器
- 相比传统多阈值结构，查表单元减少90%以上，支持混合精度
- 适合边缘设备部署，尤其在6-8段时平衡性能与成本
随着神经网络规模持续增长，低精度量化在边缘加速器中广泛应用。传统多阈值激活硬件对n比特输出需2^n个阈值，精度提升时硬件开销急剧增加。本文提出一种基于分段线性拟合的可重构激活硬件GRAU，其分段斜率近似为2的幂次。该设计仅需基本比较器和1位右移器，支持混合精度量化及SiLU等非线性函数。相比多阈值激活器，GRAU查表单元消耗降低超过90%，实现更高硬件效率、灵活性与可扩展性。通常6-8段配置能达到最佳权衡，而在高压缩低开销设置下复杂非线性可能带来更大精度损失。
原文摘要 · Abstract (English)
With the continuous growth of neural network scales, low-precision quantization is widely used in edge accelerators. Classic multi-threshold activation hardware requires 2^n thresholds for $n$-bit outputs, causing a rapid increase in hardware cost as precision increases. We propose a reconfigurable activation hardware, GRAU, based on piecewise linear fitting, where the segment slopes are approximated by powers of two. Our design requires only basic comparators and 1-bit right shifters, supporting mixed-precision quantization and nonlinear functions such as SiLU. Compared with multi-threshold activators, GRAU reduces LUT consumption by over 90%, achieving higher hardware efficiency, flexibility, and scalability. The best trade-off is usually achieved with 6-8 segments, while complex nonlinearities under aggressive low-cost settings may suffer larger accuracy degradation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。