通过博弈论视角优化梯度更新，提升神经网络抗噪能力。
Game-Theoretic Gradient Control for Robust Neural Network Training
- 将反向传播视为多智能体博弈，引入梯度丢弃机制控制梯度更新。
- 在回归任务中，梯度丢弃率p=0.9配合稳定分布扰动目标变量，显著提升抗噪性。
- 适用于需高鲁棒性的工业级表格式数据建模场景。
前馈神经网络（FFNN）对输入噪声敏感，导致预测性能下降。现有正则化方法如丢弃法常改变网络结构或忽略神经元间交互。本文通过修改反向传播，将其视为多智能体博弈，并探索受控目标变量加噪。提出“梯度丢弃”策略：在反向传播中以概率1−p随机置零隐藏层神经元梯度，同时保持前向传播活跃。该方法基于组合博弈论框架。此外，目标变量采用白噪声或稳定分布进行扰动。在十组不同表格式数据集上的实验表明，效果各异：取决于数据集与超参数设置，存在提升或削弱鲁棒性与准确率的情况。特别地，在回归任务中，梯度丢弃率p=0.9结合稳定分布目标加噪，使均方误差（MSE）曲线更平坦、对称平均绝对百分比误差（SMAPE）更稳定，显著增强输入噪声鲁棒性。结果凸显该方法潜力，强调自适应调参的关键作用，并为将神经网络视为具有涌现行为的复杂自适应系统提供了新的博弈论分析路径。
原文摘要 · Abstract (English)
Feed-forward neural networks (FFNNs) are vulnerable to input noise, reducing prediction performance. Existing regularization methods like dropout often alter network architecture or overlook neuron interactions. This study aims to enhance FFNN noise robustness by modifying backpropagation, interpreted as a multi-agent game, and exploring controlled target variable noising. Our "gradient dropout" selectively nullifies hidden layer neuron gradients with probability 1 - p during backpropagation, while keeping forward passes active. This is framed within compositional game theory. Additionally, target variables were perturbed with white noise or stable distributions. Experiments on ten diverse tabular datasets show varying impacts: improvement or diminishing of robustness and accuracy, depending on dataset and hyperparameters. Notably, on regression tasks, gradient dropout (p = 0.9) combined with stable distribution target noising significantly increased input noise robustness, evidenced by flatter MSE curves and more stable SMAPE values. These results highlight the method's potential, underscore the critical role of adaptive parameter tuning, and open new avenues for analyzing neural networks as complex adaptive systems exhibiting emergent behavior within a game-theoretic framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。