用负正则化纠正小数据下的过压缩问题
A Ridge Too Far: Correcting Over-Shrinkage via Negative Regularization
- 提出可取负值的岭回归，让弱特征方向不被过度压缩
- 实验验证负正则化能有效提升弱谱方向的模型复杂度
- 适合研究小样本建模与正则化机制的学者参考
传统正则化旨在控制方差，但在小数据回归中，当预测信号集中在受限表示的弱方向时，反而加剧欠拟合。本文研究一种具备负值能力的岭回归族，只要估计器保持良好定义，就允许可行的负区域。结果表明，负正则化在该区域起到受控的反压缩作用，尤其强化了弱特征方向的有效复杂度。基于此机制，我们形式化了弱谱欠拟合现象，推导出保守基线压缩下的符号切换结果，并研究了在整个负值可行族上基于准则的自动选择方法。合成与半合成实验验证了理论：负正则化具有可行性，能显著提升谱复杂度，表现出符号切换行为，并在预测区间有效恢复负调整。
原文摘要 · Abstract (English)
Conventional regularization is designed to control variance, but in small-data regression it can also aggravate underfitting when predictive signal is concentrated in weak directions of a restricted representation. We study a negative-capable ridge family that permits a feasible negative region whenever the estimator remains well posed, and show that negative regularization acts there as controlled anti-shrinkage by increasing effective complexity most strongly along weak eigendirections. Building on this mechanism, we formalize weak-spectrum underfitting, derive a sign-switch result under conservative baseline shrinkage, and study criterion-based automatic selection over the full negative-capable family. Synthetic and semi-synthetic experiments support the theory by verifying feasibility, spectral complexity increase, sign-switch behavior, and effective recovery of negative adjustments in the predicted regimes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。