用随机矩阵理论解析带先验的回归，揭示过拟合与正则化的关系。
Incorporating priors in learning: a random matrix study under a teacher-student framework
- 基于教师-学生框架，结合随机矩阵理论分析带高斯先验的MAP回归
- 首次给出训练与测试风险的精确渐近表达式，量化先验不匹配影响
- 可直接计算最优正则化参数，适用于有结构先验知识的场景
正则化线性回归在机器学习中至关重要，但带有信息性先验时的高维行为仍不清楚。本文首次在教师-学生框架下，对以领域知识初始化的高斯先验进行最大后验（MAP）回归，给出了训练与测试风险的精确渐近刻画。该框架统一了岭回归、最小二乘和先验引导估计器，利用随机矩阵理论导出闭式风险公式，揭示偏差-方差-先验权衡关系，解释双下降现象，并量化先验不匹配的影响。我们还找到测试风险的闭式最小化器，实现最优正则化参数的简单估计。模拟实验验证了理论的高度准确性。研究连接贝叶斯先验、经典正则化与现代渐近分析，为结构化先验知识下的学习提供概念清晰性和实践指导。
原文摘要 · Abstract (English)
Regularized linear regression is central to machine learning, yet its high-dimensional behavior with informative priors remains poorly understood. We provide the first exact asymptotic characterization of training and test risks for maximum a posteriori (MAP) regression with Gaussian priors centered at a domain-informed initialization. Our framework unifies ridge regression, least squares, and prior-informed estimators, and -- using random matrix theory -- yields closed-form risk formulas that expose the bias-variance-prior tradeoff, explain double descent, and quantify prior mismatch. We also identify a closed-form minimizer of test risk, enabling a simple estimator of the optimal regularization parameter. Simulations confirm the theory with high accuracy. By connecting Bayesian priors, classical regularization, and modern asymptotics, our results provide both conceptual clarity and practical guidance for learning with structured prior knowledge.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。