改进忆阻器计算中位置编码的数值范围，显著降低模拟-数字转换误差。
Positional Encoding in the Context of Memristor-Based Analog Computation for Automatic Speech Recognition
- 通过调节特定忆阻层的权重与精度位比例，优化模拟计算中的编码表示。
- 使执行误差降低约50%，且能耗保持稳定。
- 无需修改硬件时，移除线性变换可减少30%的误差，适合资源受限场景。
忆阻器为自然语言处理模型的资源高效计算提供了新途径，可通过模拟执行向量-矩阵乘法实现。然而，当前这些设备在权重编程和执行过程中存在较大失真。本文发现，经过变换的位置编码输出值过大，是导致忆阻器计算中模拟-数字转换（ADC）严重退化的主要原因。通过调整特定忆阻层的ADC权重与精度位比例，使执行退化相对减少约50%，同时维持估算能耗稳定。此外，在无法修改ADC的情况下，通过移除编码相关的线性变换，仍可实现约30%的退化降低。
原文摘要 · Abstract (English)
Memristors provide a new chance for resource-efficient computation of neural models for natural language processing by enabling analog execution of vector-matrix-multiplication. Yet, computations on these devices are currently subject to larger distortion, both in weight programming and execution. In this work, we identify large output values of transformed positional encodings to cause major degradation within analog-to-digital conversion (ADC) as part of memristor-based computation. By adjusting the proportion of weight and precision bits of the ADC of specific memristor layers, we reduce the degradation of the execution by ~50% relative, while keeping the estimated energy consumption stable. Additionally, we investigate scenarios where the ADC cannot be modified. In that case the degradation can be reduced by ~30% relative after removing encoding-related linear transformations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。