arXiv:2608.15602cs.LGcs.AI2026-08

通过软硬件协同设计,实现大模型超低比特推理的极致加速与节能。

FluxBin: Flexible LUT-based Ultra-low-bit LLM Inference by Algorithm-Kernel Synergy

论文配图:FluxBin: Flexible LUT-based Ultra-low-bit LLM Inference by Algorithm-Kernel Synergy
图 1 · 摘自论文原文
  • 提出解耦行列二值分解,提升表达能力并保持硬件效率。
  • 在多种模型上实现最高5.92倍加速和10.19倍能效提升。
  • 适合部署70B级大模型至单张A100显卡,内存减少4倍。

虽然二值量化理论上可为大语言模型(LLMs)带来极致压缩与加速,但现有研究常忽视专用硬件核的重要性,导致仍依赖昂贵的浮点运算或运行时反量化开销,难以释放全部加速潜力。为此,我们提出FluxBin(FLEXible LUT-based Ultra-low-bit eXecution with Binary bases),一种算法-内核协同设计方法,将后训练量化与高度优化的CUDA内核结合。算法层面,引入解耦行列二值分解以增强表示能力,同时保持硬件效率,并辅以基于海森矩阵的显著性感知混合基底以保留关键信息。内核层面,采用带缩放融合的查找表构建方法,减少浮点运算,结合虚拟列映射技术,将不规则、稀疏且显著的矩阵转换为密集执行。大量实验表明,FluxBin在多种模型架构上实现最高5.92倍加速与10.19倍能效提升,精度媲美经过大量微调的方法,成功实现70B规模模型在单张A100 GPU上的部署,内存降低4倍。代码已开源。

原文摘要 · Abstract (English)

While binary quantization theoretically promises extreme compression and acceleration for Large Language Models (LLMs), existing research often overlooks the necessity of specialized hardware kernels, thus failing to unleash the full acceleration potential due to persistent reliance on expensive floating-point arithmetic or runtime dequantization overheads. To bridge this gap, we propose FluxBin (\textbf{F}lexible \textbf{L}UT-based \textbf{U}ltra-low-bit e\textbf{X}ecution with \textbf{Bin}ary bases), an algorithm-kernel co-design that synergizes post-training quantization with a highly optimized CUDA kernel. Algorithmically, we introduce Decoupled Row-Column Binary Decomposition to enhance representational capacity while maintaining hardware efficiency, complemented by a Hessian-guided saliency-aware hybrid bases that preserve critical information. At the kernel level, we implement a Lookup Table Building Approach with Scale Fusion to reduce floating-point arithmetic, featuring a Virtual Columnar Mapping that transforms irregular, sparse, and salient matrices into dense execution. Extensive evaluations demonstrate FluxBin achieves up to $5.92\times$ speedup and $10.19\times$ energy savings across diverse model architectures, delivering comparable accuracy to heavily fine-tuned methods. This effectively enables the deployment of 70B-scale models on one single A100 GPU with a $4\times$ memory reduction. Code is available at https://github.com/nicyyyy/FluxBin.

低比特推理硬件协同大模型部署能量效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。