用椭圆算子约束损失函数,提升模型在分布外数据的泛化能力。
Elliptic Loss Regularization
- 通过椭圆算子约束损失平滑性,控制输入与损失间的映射关系。
- 在分布偏移和组不平衡任务中,显著降低测试误差。
- 方法计算高效,适用于复杂场景下的鲁棒性建模。
正则化神经网络对预测数据空间中未充分表示区域的模型行为至关重要。本文提出一种正则化技术,要求网络损失函数在数据域上满足椭圆算子条件,从而施加映射平滑性。为此,我们修改传统的经验风险最小化目标,转而最小化一个满足椭圆算子条件的新目标函数。这使得我们可以利用椭圆算子的已有理论,预测训练集外点的误差表现。本文还设计了一种可计算的近似方法,在保持计算效率的同时逼近椭圆算子的行为。最后,分析了该正则化在分布偏移和组不平衡等常见问题上的性能。数值实验验证了所提方法的有效性。
原文摘要 · Abstract (English)
Regularizing neural networks is important for anticipating model behavior in regions of the data space that are not well represented. In this work, we propose a regularization technique for enforcing a level of smoothness in the mapping between the data input space and the loss value. We specify the level of regularity by requiring that the loss of the network satisfies an elliptic operator over the data domain. To do this, we modify the usual empirical risk minimization objective such that we instead minimize a new objective that satisfies an elliptic operator over points within the domain. This allows us to use existing theory on elliptic operators to anticipate the behavior of the error for points outside the training set. We propose a tractable computational method that approximates the behavior of the elliptic operator while being computationally efficient. Finally, we analyze the properties of the proposed regularization to understand the performance on common problems of distribution shift and group imbalance. Numerical experiments confirm the utility of the proposed regularization technique.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。