首个可证明的多维超参数调优框架，解决实际中复杂调参的理论保障难题。
Provably Data-driven Multiple Hyper-parameter Tuning with Structured Loss Function
- 基于实代数几何构建多维超参数调优的通用理论框架
- 在验证损失下实现更紧致的泛化界，支持结构化数据
- 适用于加权分组Lasso等新学习任务，为自动化算法设计提供理论支撑
数据驱动的算法设计可自动调优超参数，但其统计基础仍受限于模型性能对超参数呈现隐式且高度非光滑依赖关系。现有理论仅覆盖一维（标量）超参数情形，而多维超参数调优这一重要实际问题仍未解决。本文首次建立多维超参数调优的通用理论框架，通过实代数几何工具强化半代数函数类的泛化保证，获得更紧致、更广泛适用的理论结果。同时，首次给出该设定下的下界。进一步分析在最小假设下使用验证损失进行调优的情形，并在存在额外结构时导出改进界。最后，通过新可学习性结果展示框架应用范围，包括数据驱动的加权分组Lasso与加权融合Lasso。
原文摘要 · Abstract (English)
Data-driven algorithm design automates hyperparameter tuning, but its statistical foundations remain limited because model performance can depend on hyperparameters in implicit and highly non-smooth ways. Existing guarantees focus on the simple case of a one-dimensional (scalar) hyperparameter. This leaves the practically important, multi-dimensional hyperparameter tuning setting unresolved. We address this open question by establishing the first general framework for establishing generalization guarantees for tuning multi-dimensional hyperparameters in data-driven settings. Our approach strengthens the generalization guarantee framework for semi-algebraic function classes by exploiting tools from real algebraic geometry, yielding sharper, more broadly applicable guarantees. For completeness, we also instantiate the first lower bound for this general setting. We further extend the analysis to hyperparameter tuning using the validation loss under minimal assumptions, and derive improved bounds when additional structure is available. Finally, we demonstrate the scope of the framework with new learnability results, including data-driven weighted group lasso and weighted fused lasso.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。