提出新算法c-TTv2，让模拟内存计算在迁移学习中更稳定高效。
Assessing the Performance of Analog Training for Transfer Learning
- 用截断技术改进训练算法，适应模拟器件非线性特性。
- 在CIFAR100子集上，模型准确率达78.3%，接近数字基准。
- 对器件噪声和偏差有强鲁棒性，适合真实硬件部署。
模拟内存计算是一种下一代计算范式，有望实现快速、并行且节能的深度学习训练与迁移学习（TL）。然而，由于缺乏合适的训练算法，这一潜力尚未实现。模拟存储器件存在非对称、非线性开关行为及器件间差异，导致现有大多数现成训练算法难以取得良好效果。近期提出的算法虽有所进展，但需理想对称高精度双向开关器件，且敏感度高。本文引入一种新算法c-TTv2，利用截断技术应对上述挑战。我们评估了c-TTv2在Swin-ViT模型上对CIFAR100子集进行模拟迁移学习的表现，并研究算法对权重传输噪声、对称点偏移和对称点波动等器件参数变化的鲁棒性。
原文摘要 · Abstract (English)
Analog in-memory computing is a next-generation computing paradigm that promises fast, parallel, and energy-efficient deep learning training and transfer learning (TL). However, achieving this promise has remained elusive due to a lack of suitable training algorithms. Analog memory devices exhibit asymmetric and non-linear switching behavior in addition to device-to-device variation, meaning that most, if not all, of the current off-the-shelf training algorithms cannot achieve good training outcomes. Also, recently introduced algorithms have enjoyed limited attention, as they require bi-directionally switching devices of unrealistically high symmetry and precision and are highly sensitive. A new algorithm chopped TTv2 (c-TTv2), has been introduced, which leverages the chopped technique to address many of the challenges mentioned above. In this paper, we assess the performance of the c-TTv2 algorithm for analog TL using a Swin-ViT model on a subset of the CIFAR100 dataset. We also investigate the robustness of our algorithm to changes in some device specifications, including weight transfer noise, symmetry point skew, and symmetry point variability
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。