通过可调稀疏性与隐藏比例，提升RNN在元学习中的表现
Balancing Sparse RNNs with Hyperparameterization Benefiting Meta-Learning
- 设计可变稀疏度的RNN架构，动态调整权重矩阵稀疏性
- 引入隐藏比例新指标，显著提升模型性能预测能力
- 适合关注模型内在特性与元学习优化的研究者
本文提出用于定义稀疏循环神经网络（RNN）的新超参数。这些超参数可在模型可训练权重矩阵中调节稀疏度，同时提升整体性能。该架构引入一种新度量——隐藏比例，旨在平衡模型内部未知量的分布，并具有显著的模型性能解释力。结合可变稀疏性RNN架构与隐藏比例度量，该方法在预设条件下显著提升性能表现，为通用元学习应用及基于数据集内在特征（如输入输出维度）的模型优化提供了可行路径。
原文摘要 · Abstract (English)
This paper develops alternative hyperparameters for specifying sparse Recurrent Neural Networks (RNNs). These hyperparameters allow for varying sparsity within the trainable weight matrices of the model while improving overall performance. This architecture enables the definition of a novel metric, hidden proportion, which seeks to balance the distribution of unknowns within the model and provides significant explanatory power of model performance. Together, the use of the varied sparsity RNN architecture combined with the hidden proportion metric generates significant performance gains while improving performance expectations on an a priori basis. This combined approach provides a path forward towards generalized meta-learning applications and model optimization based on intrinsic characteristics of the data set, including input and output dimensions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。