通过共享非线性操作电路，用量化提升Transformer加速效率。
QUARK: Quantization-Enabled Circuit Sharing for Transformer Acceleration by Exploiting Common Patterns in Nonlinear Operations
- 利用非线性操作中的共性模式，设计可复用的硬件电路。
- 相比以往方法，硬件开销降低超50%，端到端速度最高快1.96倍。
- 适合追求高效低功耗部署的Transformer模型开发者。
基于Transformer的模型在计算机视觉与自然语言处理领域取得了顶尖性能，但其非线性操作显著增加了推理延迟，给高效硬件加速带来挑战。为此，我们提出QUARK——一种基于量化的FPGA加速框架，通过挖掘非线性操作中的共性模式，实现高效电路共享，从而降低硬件资源需求。QUARK覆盖Transformer中所有非线性操作，采用专为加速此类运算设计的新型电路共享架构，实现高性能近似。评估表明，QUARK显著降低了主流Transformer架构中非线性算子的计算开销，在端到端性能上相比GPU实现最高提升1.96倍；同时，相较于先前方法，非线性模块的硬件开销降低超过50%，且保持高模型精度，甚至在超低比特量化下显著提升准确率。
原文摘要 · Abstract (English)
Transformer-based models have revolutionized computer vision (CV) and natural language processing (NLP) by achieving state-of-the-art performance across a range of benchmarks. However, nonlinear operations in models significantly contribute to inference latency, presenting unique challenges for efficient hardware acceleration. To this end, we propose QUARK, a quantization-enabled FPGA acceleration framework that leverages common patterns in nonlinear operations to enable efficient circuit sharing, thereby reducing hardware resource requirements. QUARK targets all nonlinear operations within Transformer-based models, achieving high-performance approximation through a novel circuit-sharing design tailored to accelerate these operations. Our evaluation demonstrates that QUARK significantly reduces the computational overhead of nonlinear operators in mainstream Transformer architectures, achieving up to a 1.96 times end-to-end speedup over GPU implementations. Moreover, QUARK lowers the hardware overhead of nonlinear modules by more than 50% compared to prior approaches, all while maintaining high model accuracy -- and even substantially boosting accuracy under ultra-low-bit quantization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。