针对阻变存储器非理想性，提出轻量微调方法实现高效模型部署。
ReRAM-aware Model Finetuning addressing I-V Non-linearity and Retention Errors
- 通过范围压缩的sinh变换缓解电流-电压非线性问题。
- 在微调中直接加入保持误差作为正则项，精度损失低于2%。
- 适合大规模模型在阻变存储器上的低开销部署，如ResNet18、DeiT-Tiny。
传统CPU、GPU和NPU架构日益受限于冯·诺依曼瓶颈。基于阻变存储器（ReRAM）交叉阵列的存内计算（IMC）提供了高密度、低功耗的替代方案，但其实际应用受制于器件非理想性。现有硬件感知训练框架通常需从头训练，对现代大模型而言计算开销过大。本文提出一种基于微调的硬件感知训练算法，实现大模型在ReRAM上的鲁棒部署且训练开销极小。方法通过范围压缩的sinh变换缓解I-V非线性，并将保持误差直接引入微调过程中的正则化损失。在图像分类与问答任务上评估，结果表明：对于ResNet18和DeiT-Tiny等大模型，精度与基线模型相当；MobileNetV3在ImageNet上精度下降不足2%；在SQuAD v2数据集上F-1分数仅下降1点。
原文摘要 · Abstract (English)
Traditional CPU, GPU, and NPU architectures are increasingly limited by the von Neumann bottleneck. While In-Memory Computing (IMC) using ReRAM crossbar arrays offers a high-density, energy-efficient alternative, its practical deployment is constrained through their non-idealities. Existing hardware-aware training frameworks often require training from scratch, which is computationally prohibitive for modern large-scale models. In this work, we propose a finetuning-based hardware-aware training algorithm that enables robust DNN deployment on ReRAM with minimal training overhead. Our approach mitigates I-V non-linearity by applying a range-shrunk sinh transformation and incorporates retention errors directly into a regularization loss during the finetuning process. We evaluate our framework across models and tasks such as image classification and question-answering (QA). Experimental results demonstrate that our method achieves similar accuracy on large-scale models like ResNet18 and DeiT-Tiny as the base model. In-case of ImageNet for MobileNetV3 families the technique has only less than 2% accuracy degradation. Further, applying the technique on the SQuAD v2 dataset results in only 1 point degradation of F-1 score.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。