为机器学习设计新型浮点格式，提升精度与效率
Novel Aspects of IEEE SA P3109 Arithmetic Formats for Machine Learning
- 定义可参数化的二进制浮点格式，支持小位宽高效表示
- 引入随机舍入与无异常操作，吞吐率显著提升
- 适合芯片厂商实现低精度加速，尤其适配神经网络
IEEE P3109草案定义了一类可参数化的二进制浮点格式及其操作，专为机器学习优化。该格式支持以少量比特位高效、一致地表示数值，参数包括位宽、精度、符号性及无穷大存在性。操作通过将浮点值解码至闭扩展实数集（包含正负无穷和NaN）来定义，显式处理NaN和无穷运算数，确保仅在真实算术中调用。支持多种舍入与饱和模式，包含随机舍入。操作无异常，通过返回值传递异常状态（如NaN），从而提升吞吐率。对共享统一比例因子的数值块操作，以底层操作统一建模。系统厂商可使用一种新型尺度不变度量——kappa近似，描述近似实现。标准函数定义及相关性质通过形式化规范自动验证与生成。
原文摘要 · Abstract (English)
The IEEE P3109 draft standard defines a parameterized family of binary floating-point formats and associated operations, with a focus on facilitating machine learning. These formats allow efficient and consistent representation of values in a small number of bits. The defined formats are parameterized over width and precision in bits, signedness, and the presence of infinities. Operations are defined by decoding floating-point values to the set of closed extended reals: the reals augmented with positive and negative infinity and NaN (Not a Number). Explicit treatment of NaN and infinite operands ensures that only real arithmetic is invoked in operation definitions. Extensive rounding and saturation modes are defined; stochastic rounding is included. Operations are exception-free, accelerating throughput, with exceptional situations communicated through return values, e.g., NaN. Operations on blocks of values sharing a common scale factor are defined in terms of the underlying operations in a uniform manner. System vendors may describe approximate implementations via a novel scale-invariant measure, akin to units in the last place, called kappa-approximation. Standard function definitions and various other properties are mechanically verified and generated using formal specifications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。