用共享尺度压缩大模型浮点数，省内存还保持精度。
Microscaling Floating Point Formats for Large Language Models
- 用块内共享尺度替代每个数值独立设尺度，实现紧凑存储。
- 在GPT-2上测试，8位微缩浮点格式训练与推理精度接近原版。
- 适合资源受限场景部署大模型，开源代码可复现。
大语言模型日益增长的计算与内存需求，亟需创新方法在不牺牲性能的前提下优化资源使用。本文提出微缩浮点格式（microscaling floating-point formats），通过在一组数值间共享比例因子，取代传统浮点中为每个值单独分配比例的做法，从而实现紧凑的单字节浮点表示，同时保持扩展动态范围。我们探索了该技术在8位浮点格式中的应用，显著降低内存占用和计算成本。在GPT-2模型架构中测试多种微缩浮点配置，结果表明其在训练与推理阶段均能实现有竞争力的准确率，验证了其作为大规模部署高效替代方案的有效性。源代码已公开于：https://github.com/unipi-dii-compressedarith/llm.c-sve。
原文摘要 · Abstract (English)
The increasing computational and memory demands of large language models (LLMs) necessitate innovative approaches to optimize resource usage without compromising performance. This paper leverages microscaling floating-point formats, a novel technique designed to address these challenges by reducing the storage and computational overhead associated with numerical representations in LLMs. Unlike traditional floating-point representations that allocate a dedicated scale for each value, microscaling employs a shared scale across a block of values, enabling compact one-byte floating-point representations while maintaining an extended dynamic range. We explore the application of microscaling in the context of 8-bit floating-point formats to significantly reduce memory footprint and computational costs. We tested several configurations of microscaling floats within the GPT-2 LLM architecture, demonstrating that microscaling data formats can achieve competitive accuracy during training and inference, proving its efficacy as a resource-efficient alternative for deploying LLMs at scale. The source code is publicly available at: https://github.com/unipi-dii-compressedarith/llm.c-sve
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。