arXiv:2609.04490cs.AIcs.LG2026-09

低精度循环网络中,状态写入规则会引发严重误差,本文提出修复方法。

When Quantization Breaks Memory: Recurrent-State Write-Back in Low-Precision Temporal Inference

论文配图:When Quantization Breaks Memory: Recurrent-State Write-Back in Low-Precision Temporal Inference
图 1 · 摘自论文原文
  • 引入状态写回机制,揭示量化存储规则对循环网络的影响
  • 4位量化使寿命参数估计误差增加70倍至300倍
  • 通过误差反馈等策略可恢复精度,无需重新训练

量化广泛用于降低神经网络推理的计算与内存开销。然而在循环网络中,量化状态需在下一时刻保存并返回,其存储规则会改变后续计算。本文提出“递归状态写回”概念,并在荧光寿命成像的GRU编码器-解码器中进行研究。核心任务是从高噪声时间分辨荧光信号中估计短寿命成分τ1和长寿命成分τ2。固定训练模型后,将连续状态传播替换为确定性4比特存储,导致τ1和τ2的估计误差分别增加约70倍和300倍。故障原因是重复微小更新低于写入阈值,使存储状态几乎不变,而网络仍持续提出修改。误差反馈、残差记忆和方向记忆能跨时间传递被抑制的更新信息,恢复精度且无需重训练。精度扫描显示提升状态精度可能恶化固定循环解,而匹配训练表明状态接口兼容性可被学习。在独立训练的LSTM中复现实验,粗粒度写回再现失败,误差反馈恢复准确率,状态特异性干预揭示细胞状态比隐藏状态更敏感。结果表明,递归状态写回是低精度循环动态的关键决定因素,状态存储接口应成为量化循环推理的核心设计考量。

原文摘要 · Abstract (English)

Quantization is widely used to reduce the computational and memory demands of neural-network inference. In recurrent networks, however, the quantized state is stored and returned at the next time step, so the rule used to store that state can alter subsequent computations. Here, we introduce recurrent-state write-back to denote this rule and isolate its effect in a compact GRU encoder--decoder for fluorescence lifetime imaging, a molecular imaging modality used in quantitative biological imaging. A central task is estimating two lifetime parameters, the short-lived component {\tau}1 and the long-lived component {\tau}2, from high-noise time-resolved fluorescence signals. Holding the trained model fixed, replacing continuous state propagation with deterministic 4-bit state storage increases estimation errors for {\tau}1 and {\tau}2 by approximately 70x and 300x, respectively. Failure occurs when repeated small updates remain below the write threshold, leaving the stored state nearly fixed while the network continues to propose change. Error feedback, residual memory, and direction memory carry information from these suppressed updates across time and recover accuracy without retraining. Precision sweeps show that increasing state precision can worsen a fixed recurrent solution, while matched training shows that compatibility with the state interface can be learned. To test whether this behavior extends beyond the GRU, we repeat the post-training intervention in an independently trained LSTM, where coarse write-back reproduces the failure, error feedback restores accuracy, and state-specific interventions reveal greater sensitivity of the cell state than the hidden state. Our results establish recurrent-state write-back as a key determinant of low-precision recurrent dynamics and identify the state-storage interface as a central design consideration for quantized recurrent inference.

量化循环网络状态存储生物成像

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。