混合实数与复数神经网络,提升信号处理效率与精度
Hybrid Real- and Complex-valued Neural Network Architecture
- 构建含实/复数双路径的神经网络,通过域转换函数互通信息
- 在AudioMNIST上参数更少、交叉熵更低,优于纯实数网络
- 适合需要高效处理复数信号的领域,如语音与雷达信号处理
我们提出一种混合实数与复数神经网络(HNN)架构,结合实数运算的计算效率与对复数数据的有效处理能力。通过实验揭示纯实数神经网络(RVNN)在处理固有复数问题时存在显著效率瓶颈,尽管可学习复数卷积但受限于实数结构。HNN采用包含实/复数路径的模块化设计,通过域转换函数实现跨域信息交换,并引入新型复数激活函数,具有更强泛化能力与参数效率。还提出HNN专用的架构搜索方法以应对更大的解空间。在AudioMNIST数据集上的实验表明,无论何种情况,HNN均在保持更低交叉熵损失的同时,使用更少参数,验证了部分复数处理在神经网络中的潜力,为多信号处理领域提供了新范式。
原文摘要 · Abstract (English)
We propose a \emph{hybrid} real- and complex-valued \emph{neural network} (HNN) architecture, designed to combine the computational efficiency of real-valued processing with the ability to effectively handle complex-valued data. We illustrate the limitations of using real-valued neural networks (RVNNs) for inherently complex-valued problems by showing how it learnt to perform complex-valued convolution, but with notable inefficiencies stemming from its real-valued constraints. To create the HNN, we propose to use building blocks containing both real- and complex-valued paths, where information between domains is exchanged through domain conversion functions. We also introduce novel complex-valued activation functions, with higher generalisation and parameterisation efficiency. HNN-specific architecture search techniques are described to navigate the larger solution space. Experiments with the AudioMNIST dataset demonstrate that the HNN reduces cross-entropy loss and consumes less parameters compared to an RVNN for all considered cases. Such results highlight the potential for the use of partially complex-valued processing in neural networks and applications for HNNs in many signal processing domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。