arXiv:2509.16989cs.LGcs.AI2025-09被引 9

将大模型权重分解为三元平面,实现高效低比特推理。

PTQTP: Post-Training Quantization to Trit-Planes for Large Language Models

  • 将权重拆分为三元平面与尺度,实现免乘法的加法推理。
  • 0.6B-70B模型上性能逼近1.58比特训练方法,仅需1小时量化。
  • 无需修改结构,统一三元运算,适合边缘设备部署。

大语言模型的后训练低比特量化仍面临计算效率与表征能力之间的根本矛盾。现有超低比特方法依赖二值近似或量化感知训练,常受限于表征能力或训练开销巨大。本文提出PTQTP,一种结构化后训练量化框架,将权重矩阵分解为双三元{-1, 0, 1}三元平面。该方法通过解耦权重的离散拓扑(三元平面)与连续幅度(缩放系数),实现无乘法的加法推理,有效支持高保真稀疏近似。PTQTP具备:(1) 理论驱动的渐进式近似算法,保障全局权重一致性;(2) 无需架构修改的模型无关部署;(3) 统一三元运算,消除混合精度开销。在LLaMA3.x和Qwen3(0.6B-70B)上的全面实验表明,PTQTP显著优于现有亚4比特后训练方法,在语言推理、数学推理及编程任务上表现优异。其性能媲美1.58比特量化感知训练,但仅需单小时量化,远低于训练方法所需的10-14 GPU天。端到端推理速度达FP16基线的4.63倍,为资源受限环境下的高效LLM部署提供了新范式。代码将发布于https://github.com/HeXiao-55/PTQTP。

原文摘要 · Abstract (English)

Post-training quantization (PTQ) of large language models (LLMs) to extremely low bit-widths remains challenging due to the fundamental trade-off between computational efficiency and representational capacity. While existing ultra-low-bit methods rely on binary approximations or quantization-aware training(QAT), they often suffer from either limited representational capacity or huge training resource overhead. We introduce PTQ to Trit-Planes (PTQTP), a structured PTQ framework that decomposes weight matrices into dual ternary {-1, 0, 1} trit-planes. This approach achieves multiplication-free additive inference by decoupling weights into discrete topology (trit-planes) and continuous magnitude (scales), effectively enabling high-fidelity sparse approximation. PTQTP provides: (1) a theoretically grounded progressive approximation algorithm ensuring global weight consistency; (2) model-agnostic deployment without architectural modifications; and (3) uniform ternary operations that eliminate mixed-precision overhead. Comprehensive experiments on LLaMA3.x and Qwen3 (0.6B-70B) demonstrate that PTQTP significantly outperforms sub-4bit PTQ methods on both language reasoning tasks and mathematical reasoning as well as coding. PTQTP rivals the 1.58-bit QAT performance while requiring only single-hour quantization compared to 10-14 GPU days for training-based methods, and the end-to-end inference speed achieves 4.63$\times$ faster than the FP16 baseline model, establishing a new and practical solution for efficient LLM deployment in resource-constrained environments. Code will available at https://github.com/HeXiao-55/PTQTP.

量化大模型低比特推理加速

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。