arXiv:2602.21321cs.LGcs.AR2026-02

提出动态追踪权重对称点的新方法,解决模拟存内计算中的训练偏差问题。

Dynamic Symmetric Point Tracking: Tackling Non-ideal Reference in Analog In-memory Training

  • 训练中动态估计设备对称点,避免预校准带来的误差
  • 理论证明方法收敛性,脉冲复杂度更低且误差更小
  • 适合资源受限的边缘AI部署,尤其适用于存内计算芯片

模拟存内计算(AIMC)在阻变交叉阵列中直接执行计算,为扩展大模型提供了节能平台。然而,非理想模拟器件特性使训练困难,其更新不对称会导致权重更新系统性漂移至设备特异的对称点(SP),而该点通常不对应于训练目标的最优解。现有方法假设已知SP,通过设置参考点为SP进行预校准,但校准需大量脉冲更新,且残余误差会直接影响训练性能。本文首次理论分析了SP校准的脉冲复杂度与估计误差,并提出一种训练期间动态追踪SP的方法,建立了其收敛性保证。此外,结合数字信号处理中的斩波与滤波技术,设计了增强版本。数值实验表明,该方法在效率和效果上均优于传统方案。

原文摘要 · Abstract (English)

Analog in-memory computing (AIMC) performs computation directly within resistive crossbar arrays, offering an energy-efficient platform to scale large vision and language models. However, non-ideal analog device properties make the training on AIMC devices challenging. In particular, its update asymmetry can induce a systematic drift of weight updates towards a device-specific symmetric point (SP), which typically does not align with the optimum of the training objective. To mitigate this bias, most existing works assume the SP is known and pre-calibrate it to zero before training by setting the reference point as the SP. Nevertheless, calibrating AIMC devices requires costly pulse updates, and residual calibration error can directly degrade training performance. In this work, we present the first theoretical characterization of the pulse complexity of SP calibration and the resulting estimation error. We further propose a dynamic SP estimation method that tracks the SP during model training, and establishes its convergence guarantees. In addition, we develop an enhanced variant based on chopping and filtering techniques from digital signal processing. Numerical experiments demonstrate both the efficiency and effectiveness of the proposed method.

存内计算模拟硬件神经网络训练边缘AI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。