用激活加权的残差编码修复低比特量化模型，显著提升性能。
Activation-Weighted Seeded Residual Coding for Low-Bit LLM Weight Repair
- 基于种子生成基向量，用低比特系数编码权重残差。
- 在Qwen2.5-3B上修复后，困惑度降低88.2%，准确率提升71.3%。
- 仅需0.8%模型大小的附加开销，适合部署优化场景。
低比特权重量化虽节省存储，但引入误差导致语言模型质量下降。本文提出激活加权种子残差编码（AWSRC），作为现有量化骨干的紧凑修复编解码器。给定重构权重 $W_0$，AWSRC 使用确定性种子生成的基向量对残差 $W-W_0$ 进行编码。侧车（sidecar）存储种子选择器、低比特系数和缩放因子，而非显式码本。激活统计信息优先修复影响层输出的误差。在 Qwen2.5-3B-Instruct 上，向 INT4 RTN 骨干添加 0.162 比特/权重的修复空间，可关闭 88.2%、78.9% 和 71.3% 的匹配困惑度、KL 散度和准确率差距至 BF16 基线。修复一个强低比特骨干也提升了所有测量指标。使用约 49.25 MB 的侧车（占 BF16 模型权重的 0.8%），AWSRC 在稀疏、低秩和向量量化编解码器中实现了最佳困惑度与平均任务准确率。
原文摘要 · Abstract (English)
Low-bit weight quantization saves storage but leaves errors that degrade language-model quality. We introduce Activation-Weighted Seeded Residual Coding (AWSRC), a compact repair codec for an existing quantization backbone. Given a reconstructed weight $W_0$, AWSRC encodes the residual $W-W_0$ using deterministic seed-generated bases. The sidecar stores seed selectors, low-bit coefficients, and scales rather than an explicit codebook. Activation statistics prioritize errors that affect layer outputs. On Qwen2.5-3B-Instruct, adding 0.162 scope-bits/weight to an INT4 RTN backbone closes 88.2%, 78.9%, and 71.3% of the matched PPL, KL, and accuracy gaps to BF16. Repairing a matched strong low-bit backbone also improves all measured quality metrics. With a matched 49.25 MB sidecar, about 0.8% of the BF16 model-weight payload, AWSRC gives the best perplexity and mean task accuracy among sparse, low-rank, and vector-quantized codecs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。