将神经元权重拆分为正负两部分，提升模型抗噪与泛化能力。
Revisiting Gradient Descent: A Dual-Weight Method for Improved Learning
- 权重分解为正向与负向两部分独立更新
- 在稀疏或噪声数据下显著减少过拟合
- 保持原有推理开销，适合实际部署
我们提出一种新型神经网络学习框架，将每个神经元的权重向量分解为两个独立部分 $W_1$ 与 $W_2$，直接在神经元层面建模对比信息。传统梯度下降将正例（目标）与负例（非目标）特征混合存储于单一权重向量中，常掩盖细粒度差异。本方法通过分别更新目标与非目标特征，最终形成有效权重 $W = W_1 - W_2$，增强了对噪声和类别不平衡的鲁棒性。在回归任务（加州房价、葡萄酒品质）与分类任务（MNIST、Fashion-MNIST、CIFAR-10）上的实验表明，该方法显著提升泛化性能，尤其在训练数据稀疏或含噪时表现更优。关键优势在于推理复杂度与标准 $WX + \text{bias}$ 一致，无需额外计算开销，为高效学习提供实用方案。
原文摘要 · Abstract (English)
We introduce a novel framework for learning in neural networks by decomposing each neuron's weight vector into two distinct parts, $W_1$ and $W_2$, thereby modeling contrastive information directly at the neuron level. Traditional gradient descent stores both positive (target) and negative (non-target) feature information in a single weight vector, often obscuring fine-grained distinctions. Our approach, by contrast, maintains separate updates for target and non-target features, ultimately forming a single effective weight $W = W_1 - W_2$ that is more robust to noise and class imbalance. Experimental results on both regression (California Housing, Wine Quality) and classification (MNIST, Fashion-MNIST, CIFAR-10) tasks suggest that this decomposition enhances generalization and resists overfitting, especially when training data are sparse or noisy. Crucially, the inference complexity remains the same as in the standard $WX + \text{bias}$ setup, offering a practical solution for improved learning without additional inference-time overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。