提出新型4位浮点格式HiFloat4，提升语言模型推理精度与效率
HiFloat4 Format for Language Model Inference
- 64个4比特元素共享32位缩放元数据，实现高效存储
- 在多个大模型上平均精度优于当前最优的NVFP4格式
- 适合需要高能效推理的边缘设备部署
本文提出针对深度学习优化的块浮点数据格式HiFloat4（HiF4），每个单元打包64个4比特元素，共享32位缩放元数据，平均每值4.5比特。该格式采用三级缩放层次结构，有效捕捉组间与组内动态范围，提升表示空间利用率。由于64元素的大分组规模，可支持高度固定点化的矩阵乘法，显著降低硬件面积与功耗。我们在LLaMA、Qwen、Mistral、DeepSeek-V3.1和LongCat等多个语言模型上进行推理实验，结果表明，HiF4在多种模型和下游任务中平均精度均高于当前最优的NVFP4格式。
原文摘要 · Abstract (English)
This paper introduces HiFloat4 (HiF4), a block floating-point data format tailored for deep learning. Each HiF4 unit packs 64 4-bit elements with 32 bits of shared scaling metadata, averaging 4.5 bits per value. The metadata specifies a three-level scaling hierarchy, capturing inter- and intra-group dynamic range while improving the utilization of the representational space. In addition, the large 64-element group size enables matrix multiplications to be executed in a highly fixed-point manner, significantly reducing hardware area and power consumption. To evaluate the proposed format, we conducted inference experiments on several language models, including LLaMA, Qwen, Mistral, DeepSeek-V3.1 and LongCat. Results show that HiF4 achieves higher average accuracy than the state-of-the-art NVFP4 format across multiple models and diverse downstream tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。