对比经典与贝叶斯稀疏回归在高维弱信号下的表现，揭示各方法优劣。
Sparse Regression under Correlation and Weak Signals: A Reproducible Benchmark of Classical and Bayesian Methods
- 构建合成数据集，测试六种方法在不同相关性、信噪比和维度下的表现
- 贝叶斯方法预测误差更低（MSE 72 vs. 108-267），Horseshoe覆盖率达94.8%
- Lasso在变量选择上与Spike-and-Slab相当，适合无需后验信息的场景
选择经典或贝叶斯稀疏回归方法存在权衡：如Lasso等惩罚估计器可在毫秒级完成，但无法提供不确定性估计；而Horseshoe和Spike-and-Slab先验虽能生成完整后验分布，但需数分钟每轮采样。然而，很少有研究在真实困难条件下——特征相关、信号微弱、维度增长——对两类方法进行直接比较。本文在包含三种协方差结构（ρ最高达0.9）、四种信噪比水平及p∈{20,50,100}的合成数据上，对六种方法（OLS、Ridge、Lasso、Elastic Net、Horseshoe、Spike-and-Slab）进行了基准测试，涵盖糖尿病数据集，共超过2600次实验。结果表明，贝叶斯方法在预测误差上更优（MSE 72 vs. 108–267），Horseshoe实现接近名义95%覆盖率（94.8%）；而Spike-and-Slab虽区间更窄，但覆盖率仅91.9%，其连续松弛可能是原因。变量选择方面，Lasso与Spike-and-Slab F1得分均约0.47，当无需后验信息时，Lasso为实用首选。代码与数据见https://github.com/xiao98/sparse-bayesian-regression-bench。
原文摘要 · Abstract (English)
Choosing between classical and Bayesian sparse regression methods involves a real trade-off: penalized estimators like Lasso run in milliseconds but give no uncertainty estimates,while Horseshoe and Spike-and-Slab priors produce full posteriors but need MCMC chains that take minutes per fit.Surprisingly few studies compare these two families head-to-head under the conditions that actually make sparse regression hard -- correlated features, weak signals, and growing dimensionality. We benchmark six methods (OLS, Ridge,Lasso, Elastic Net, Horseshoe, Spike-and-Slab) on synthetic data with three covariance structures (rho up to 0.9), four SNR levels, and p in {20, 50, 100}, plus the Diabetes dataset,totalling over 2,600 experiments. The results are clear on some points and nuanced on others. Bayesian methods win on prediction error (MSE 72 vs. 108-267), and the Horseshoe delivers near-nominal 95% coverage (94.8%). But Spike-and-Slab,despite narrower intervals, under-covers at 91.9% -- its continuous relaxation likely plays a role. For variable selection, Lasso and Spike-and-Slab tie at F1 ~ 0.47, making Lasso the practical default when posteriors are not needed. Code and data are available at https://github.com/xiao98/sparse-bayesian-regression-bench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。