通过统计检验方法提升模型超参数选择的可靠性，保障实际部署效果。
Ensuring Reliability via Hyperparameter Selection: Review and Advances
- 将超参数选择建模为多重假设检验，提供统计保证
- 可确保选中参数在真实数据上表现稳定，避免过拟合风险
- 适合对可靠性要求高的工程场景，如通信系统
超参数选择是人工智能模型部署中的关键步骤，尤其在当前大模型和预训练模型盛行的背景下。本文将超参数选择问题形式化为多重假设检验，使研究者能够对选定超参数所达成的总体风险度量提供统计保证。论文综述了‘学后测试’（Learn-Then-Test, LTT）框架，并探讨了多个面向工程实际的应用扩展：包括不同风险度量与统计保证、多目标优化、引入先验知识与参数依赖结构，以及自适应机制。文中还展示了该方法在通信系统中的具体应用实例。
原文摘要 · Abstract (English)
Hyperparameter selection is a critical step in the deployment of artificial intelligence (AI) models, particularly in the current era of foundational, pre-trained, models. By framing hyperparameter selection as a multiple hypothesis testing problem, recent research has shown that it is possible to provide statistical guarantees on population risk measures attained by the selected hyperparameter. This paper reviews the Learn-Then-Test (LTT) framework, which formalizes this approach, and explores several extensions tailored to engineering-relevant scenarios. These extensions encompass different risk measures and statistical guarantees, multi-objective optimization, the incorporation of prior knowledge and dependency structures into the hyperparameter selection process, as well as adaptivity. The paper also includes illustrative applications for communication systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。