用1比特符号+轴向缩放因子，高效压缩大模型微调增量。
Per-Axis Weight Deltas for Frequent Model Updates
- 1比特存储权重差值符号，搭配行列级浮点缩放因子。
- 相比标量缩放，重建误差降低3倍以上，仅需少量校准数据。
- 支持快速加载，适合频繁更新的垂直领域大模型部署。
服务多个任务专用的大语言模型变体常受限于微调检查点过大和冷启动延迟。由于微调权重与基础模型差异较小且具有结构化特征，一种自然思路是将其表示为压缩的增量。本文提出一种简单的1比特增量方案：仅存储权重差值的符号，并配合从少量校准数据中学习的轻量级逐轴（行/列）FP16缩放因子。该设计在保持1比特压缩性的同时，更准确地捕捉了权重维度间的差异，重建质量显著优于标量方法。系统层面，通过单次操作传输打包后的增量，大幅降低冷启动延迟和存储开销，生成的模型体积仅为全精度FP16检查点的几分之一。该方法可直接插入现有流程，校准数据需求少，且避免密集重构，保障推理效率。实验代码与设置已开源于https://github.com/kuiumdjiev/Per-Axis-Weight-Deltas-for-Frequent-Model-Updates。
原文摘要 · Abstract (English)
Serving many task-specialized LLM variants is often limited by the large size of fine-tuned checkpoints and the resulting cold-start latency. Since fine-tuned weights differ from their base model by relatively small structured residuals, a natural approach is to represent them as compressed deltas. We propose a simple 1-bit delta scheme that stores only the sign of the weight difference together with lightweight per-axis (row/column) FP16 scaling factors, learned from a small calibration set. This design preserves the compactness of 1-bit deltas while more accurately capturing variation across weight dimensions, leading to improved reconstruction quality over scalar alternatives. From a systems perspective, a streamlined loader that transfers packed deltas in a single operation per module reduces cold-start latency and storage overhead, with artifacts several times smaller than a full FP16 checkpoint. The method is drop-in, requires minimal calibration data, and maintains inference efficiency by avoiding dense reconstruction. Our experimental setup and source code are available at https://github.com/kuiumdjiev/Per-Axis-Weight-Deltas-for-Frequent-Model-Updates.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。