低秩B样条模型通过局部能量正则化提升抗过拟合能力。
Penalizing Localized Dirichlet Energies in Low Rank Tensor Products
- 用小超立方体定义局部Dirichlet能量，替代全局正则化。
- 在多数数据集上，低秩B样条比神经网络更抗过拟合。
- 适合需要稳健回归与小样本建模的场景。
我们研究用于回归任务的低秩张量积B样条（TPBS）模型，并探讨Dirichlet能量作为平滑性度量。我们证明了TPBS模型的Dirichlet能量存在闭式表达，且在某些情况下可实现完美插值并保持指数级小的能量。这表明基于全局Dirichlet能量的正则化无效。为此，我们提出一种新正则化策略，基于以训练点为中心的小超立方体上的局部Dirichlet能量。借助预训练的TPBS模型，我们还引入两种从不完整样本中推断的估计器。与神经网络的对比实验表明，在大多数数据集上，TPBS模型在过拟合区域表现优于神经网络，其他情况下也保持竞争力。总体而言，TPBS模型对过拟合更具鲁棒性，且始终受益于正则化，而神经网络更易过拟合，正则化效果较弱。
原文摘要 · Abstract (English)
We study low-rank tensor-product B-spline (TPBS) models for regression tasks and investigate Dirichlet energy as a measure of smoothness. We show that TPBS models admit a closed-form expression for the Dirichlet energy, and reveal scenarios where perfect interpolation is possible with exponentially small Dirichlet energy. This renders global Dirichlet energy-based regularization ineffective. To address this limitation, we propose a novel regularization strategy based on local Dirichlet energies defined on small hypercubes centered at the training points. Leveraging pretrained TPBS models, we also introduce two estimators for inference from incomplete samples. Comparative experiments with neural networks demonstrate that TPBS models outperform neural networks in the overfitting regime for most datasets, and maintain competitive performance otherwise. Overall, TPBS models exhibit greater robustness to overfitting and consistently benefit from regularization, while neural networks are more sensitive to overfitting and less effective in leveraging regularization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。