用神经速度动态调参，无需验证集也能高效训练
Neural Velocity for hyperparameter tuning
- 引入神经速度衡量神经元变化速率，替代验证损失
- 可仅通过前向噪声采样实现，降低对验证集依赖
- 适合资源有限或难以获取验证数据的场景
超参数调优（如学习率衰减、停止准则设定）通常依赖验证损失监控。本文提出NeVe，一种基于新概念‘神经速度’的动态训练方法。神经速度衡量每个神经元转移函数的变化速率，是模型收敛的指示器：通过在网络中前向传播噪声即可采样神经速度，减少对独立验证集的需求。实验表明，神经速度可作为优化神经网络训练效率的关键指标。
原文摘要 · Abstract (English)
Hyperparameter tuning, such as learning rate decay and defining a stopping criterion, often relies on monitoring the validation loss. This paper presents NeVe, a dynamic training approach that adjusts the learning rate and defines the stop criterion based on the novel notion of "neural velocity". The neural velocity measures the rate of change of each neuron's transfer function and is an indicator of model convergence: sampling neural velocity can be performed even by forwarding noise in the network, reducing the need for a held-out dataset. Our findings show the potential of neural velocity as a key metric for optimizing neural network training efficiently
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。