用神经网络自适应调节变量惩罚，提升结构化高维回归精度
Nash: Neural Adaptive Shrinkage for Structured High-Dimensional Regression
- 通过神经网络按变量动态调整正则化强度
- 比传统方法快74至106倍，且无需交叉验证调参
- 适合有领域信息的生物医学等复杂数据建模
稀疏线性回归是数据分析的基础工具。但当协变量具有结构或来自异质来源时，传统方法表现不佳。在生物医学应用中，协变量可能来自不同模态或具有图结构。我们提出神经自适应收缩（Nash），一种统一框架，利用神经网络将协变量特异性辅助信息融入稀疏回归。Nash能自适应地在每个协变量上调节惩罚，学习定制正则化而无需交叉验证。采用分裂变分经验贝叶斯算法，将先验学习与后验推断解耦，使每轮迭代的M步从每轮需$ℂ(p)$次神经网络遍历降至单次批量处理，相比之前提出的坐标上升CAVI算法，在$10^2$到$10^4$之间的$ p $值下实现74至106倍的墙钟速度提升。真实数据实验表明，Nash在准确性和适应性上优于现有方法。
原文摘要 · Abstract (English)
Sparse linear regression is a fundamental tool in data analysis. However, traditional approaches often fall short when covariates exhibit structure or arise from heterogeneous sources. In biomedical applications, covariates may stem from distinct modalities or be structured according to an underlying graph. We introduce \textit{Neural Adaptive Shrinkage} (Nash), a unified framework that integrates covariate-specific side information into sparse regression via neural networks. Nash adaptively modulates penalties on a per-covariate basis, learning to tailor regularization without cross-validation. We use a \textit{split variational empirical Bayes} algorithm that decouples prior learning from posterior inference, reducing the M-step from $\mathcal{O}(p) $ neural-network passes per sweep to a single batched pass, a \textit{74 to 106x wall-clock speedup} over previously proposed coordinate ascent CAVI for p between $10^2$ and $10^4$. Experiments on real data demonstrate that Nash improves accuracy and adaptability over existing methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。