提出稀疏激活神经算子，实现低延迟高能效的边缘虚拟传感。
Low-Latency Activation-Regularized Sparse Neural Operators with Distillation Assistance Towards Real-Time Edge-Deployable Virtual Sensing
- 设计SAR层，单步训练实现激活稀疏化，兼容事件驱动计算。
- 在热交换器数据集上，误差降低两到七倍，能效提升超五倍。
- 适合追求低延迟、低功耗的边缘智能与类脑计算系统部署。
虚拟传感使数字孪生与安全关键系统能够实时重构和预测时空物理过程。然而，传统计算与数据驱动方法在泛化能力、延迟和能效方面面临挑战，难以在边缘设备部署。神经算子虽具前景，但仍依赖高功耗硬件。脉冲神经元与类脑计算可提升效率，但代理梯度训练与多步脉冲引入收敛与延迟难题。本文提出稀疏激活ReLU（SAR）层，无需代理梯度训练即可实现激活稀疏化，且兼容事件驱动计算。在基于主干网络的NOMAD架构中，SAR相较可变脉冲神经元（VSN）与漏电积分-发放（LIF）模型，在联合延迟-误差-能量（LEE）指标上提升超五倍。通过分析脉冲熵与特征使用情况，并引入合成知识蒸馏，进一步将LEE得分降低逾两倍。最后，通过基于ReLU的脉冲损失与图邻域阈值法改进VSN，分别在热交换器数据集上将L2误差降低两倍与近七倍，同时减少脉冲与空间聚合。整体工作为面向能效的虚拟传感提供新框架，适用于类脑或边缘设备集成，可作为未来高效稀疏或类脑模型的性能基准。
原文摘要 · Abstract (English)
Virtual sensing enables digital twins and safety-critical systems to reconstruct and forecast spatial-temporal physics in real time. However, conventional computational and data-driven methods often face challenges in generalization, latency, and energy efficiency for edge deployment. Neural operators offer a promising alternative but remain reliant on power-intensive hardware. Spiking neurons and neuromorphic computing can improve efficiency, yet surrogate-gradient training and multi-step spiking introduce convergence and latency challenges. We propose the Sparse-Activation-ReLU (SAR) layer, a single-step alternative that promotes activation sparsity without surrogate-gradient training while remaining compatible with event-based computing. Within a trunk-based NOMAD architecture, SAR achieves over a fivefold improvement in the combined Latency-Error-Energy (LEE) metric compared with Variable Spiking Neuron (VSN) and Leaky Integrate-and-Fire (LIF) implementations. We further analyze spiking entropy and feature usage and introduce synthetic knowledge distillation, reducing the LEE score by more than twofold. Finally, we improve VSN through a ReLU-based spiking loss and graph-neighbor thresholding. On the Heat Exchanger dataset, these approaches reduce L2 error by more than twofold and nearly sevenfold, respectively, while reducing spiking and spatial aggregation. Overall, the work presented is a step towards energy-efficient virtual sensing by providing an alternative framework that can be positioned towards neuromorphic or other edge device integration that can be a gold standard to compare latency, energy, and error performance for future efficient designs that are sparsity or brain-inspired spiking based.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。