为岭回归设计无需分布假设的置信椭球，可量化参数不确定性
Distribution-Free Confidence Ellipsoids for Ridge Regression with PAC Bounds
- 基于SPS算法扩展，构建岭回归的非渐近置信椭球
- 首次给出正则化参数对椭球大小影响的精确上界
- 适用于数据激励不足场景，适合控制与信号处理领域
线性参数模型广泛应用于控制与信号处理中，最小二乘法是典型求解方式。当输入激励不足时，最小二乘问题可能无解或数值不稳定。可通过正则化（如岭回归）解决此问题。尽管正则化能降低方差误差，但仍需量化估计不确定性。一种可行方法是使用符号扰动和算法（SPS）的椭球外逼近（EOA）构建置信椭球，该方法在噪声独立且关于零对称的假设下给出非渐近置信区域。本文将SPS EOA算法扩展至岭回归，并推导出结果区域大小的概率近似正确（PAC）上界。相比以往分析，本工作明确揭示了正则化参数对区域大小的影响，并在更弱的激励条件下提供更紧的上界。最后，通过仿真实验验证了正则化在实际中的效果。
原文摘要 · Abstract (English)
Linearly parametrized models are widely used in control and signal processing, with the least-squares (LS) estimate being the archetypical solution. When the input is insufficiently exciting, the LS problem may be unsolvable or numerically unstable. This issue can be resolved through regularization, typically with ridge regression. Although regularized estimators reduce the variance error, it remains important to quantify their estimation uncertainty. A possible approach for linear regression is to construct confidence ellipsoids with the Sign-Perturbed Sums (SPS) ellipsoidal outer approximation (EOA) algorithm. The SPS EOA builds non-asymptotic confidence ellipsoids under the assumption that the noises are independent and symmetric about zero. This paper introduces an extension of the SPS EOA algorithm to ridge regression, and derives probably approximately correct (PAC) upper bounds for the resulting region sizes. Compared with previous analyses, our result explicitly show how the regularization parameter affects the region sizes, and provide tighter bounds under weaker excitation assumptions. Finally, the practical effect of regularization is also demonstrated via simulation experiments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。