用符号代替浮点数，让设备端微调模型更小更快。
Signs Beat Floats: Low-Rank Double-Binary Adaptation for On-Device Fine-Tuning
- 用二值符号替代低秩矩阵中的浮点数，仅保留通道缩放量
- 在相同模型大小下优于低比特基线，且在特定条件下达到全精度质量
- 适配设备端部署，内存占用减少10倍以上，延迟增加不足8%
设备端大模型微调通常保持量化基础模型冻结，仅训练和部署小型任务专用的LoRA适配器。在未合并适配器模式下，适配器不仅是紧凑存储模块，还引入额外的密集浮点分支，维持可训练状态，并作为通信与热切换单元。我们提出LoRDBA，一种兼容LoRA的适配器，将两个低秩因子均替换为二值符号载体，同时通过轻量级通道级缩放表示幅度，将密集适配器分支转化为两个符号累加矩阵乘法，中间穿插通道级缩放。有限样本分析表明，重构质量由原始LoRA因子的残差与幅度比决定。在适配器模式实验中，LoRDBA在匹配模型尺寸下优于低比特基线，且在部分场景下达到fp16 LoRA性能。即使在未合并模式下，当秩r=16时，预填充延迟最多增加8%，而适配器体积减少超过10倍，训练内存开销约为fp16 LoRA的1.6倍。
原文摘要 · Abstract (English)
On-device adaptation of large language models commonly keeps a quantized base model frozen while training and deploying a small, task-specific LoRA adapter. In the unmerged adapter-mode setting, however, the adapter is more than a compact storage module; it introduces an additional dense floating-point branch, maintains a trainable state for local updates, and acts as a unit of communication and hot-swapping.We introduce LoRDBA, a LoRA-compatible adapter that replaces both low-rank factors with binary sign carriers while representing magnitudes through lightweight, channel-wise scales, converting the dense adapter branch into two sign-accumulation matrix multiplications interleaved with channel-wise scaling. A finite-sample analysis shows that reconstruction quality is governed by the residual-to-magnitude ratio of the original LoRA factors. In adapter-mode experiments, LoRDBA outperforms low-bit baselines at matched model sizes while matching fp16 LoRA quality in selected regimes. The unmerged adapter incurs at most 8% prefill latency overhead at matched rank r=16 despite an over 10x reduction in adapter footprint, with moderate training memory overhead of approximately 1.6x that of fp16 LoRA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。