用可学习的随机矩阵元素调制，大幅减少全连接层参数量
Element-wise Modulation of Random Matrices for Efficient Neural Layers
- 用固定随机矩阵加轻量可学习参数实现特征混合与适配解耦
- 参数量降为线性规模，多个基准测试保持可靠精度
- 适合资源受限场景下的模型部署与架构扩展
全连接层因密集且常冗余的参数化，成为深度神经网络中内存和计算开销的主要来源。尽管存在多种压缩技术，但往往引入复杂的工程权衡或损害模型性能。本文提出参数化随机投影（PRP）层，通过使用固定随机矩阵并由轻量级可学习元素级参数进行调制，将特征混合与适应过程解耦。该架构将可训练参数数量降至线性量级，同时在多个基准测试中保持可靠的准确率。设计提供了一种稳定、计算高效的解决方案，适用于模型架构扩展及资源受限环境下的部署。
原文摘要 · Abstract (English)
Fully connected layers are a primary source of memory and computational overhead in deep neural networks due to their dense, often redundant parameterization. While various compression techniques exist, they frequently introduce complex engineering trade-offs or degrade model performance. We propose the Parametrized Random Projection (PRP) layer, a novel approach that decouples feature mixing from adaptation by utilizing a fixed random matrix modulated by lightweight, learnable element-wise parameters. This architecture drastically reduces the trainable parameter count to a linear scale while retaining reliable accuracy across various benchmarks. The design serves as a stable, computationally efficient solution for architectural scaling and deployment in resource-limited settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。