用稀疏信号处理视角揭示深度网络隐藏的凸性，助理解与优化。
Unveiling Hidden Convexity in Deep Learning: a Sparse Signal Processing Perspective
- 从稀疏建模出发，将ReLU网络损失转化为可解凸问题
- 证明两层网络在特定条件下可全局优化，避免局部最优
- 适合对理论深度学习和信号处理交叉感兴趣的读者
深度神经网络（DNNs），尤其是使用修正线性单元（ReLU）激活函数的网络，在图像识别、音频处理和语言建模等任务中取得了显著成功。然而，其损失函数的非凸性给优化带来了挑战，并限制了理论理解。本文强调，近期发现的ReLU网络与稀疏信号处理模型之间的凸等价关系，可有效应对训练与理解难题。研究表明，某些网络架构（如两层ReLU网络及其他更深或不同结构）的损失景观中存在隐藏凸性。本文旨在提供一个易懂且教育性的综述，连接深度学习数学前沿与传统信号处理，推动更广泛的应用。
原文摘要 · Abstract (English)
Deep neural networks (DNNs), particularly those using Rectified Linear Unit (ReLU) activation functions, have achieved remarkable success across diverse machine learning tasks, including image recognition, audio processing, and language modeling. Despite this success, the non-convex nature of DNN loss functions complicates optimization and limits theoretical understanding. In this paper, we highlight how recently developed convex equivalences of ReLU NNs and their connections to sparse signal processing models can address the challenges of training and understanding NNs. Recent research has uncovered several hidden convexities in the loss landscapes of certain NN architectures, notably two-layer ReLU networks and other deeper or varied architectures. This paper seeks to provide an accessible and educational overview that bridges recent advances in the mathematics of deep learning with traditional signal processing, encouraging broader signal processing applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。