动态调整超参重要性，加速多目标优化收敛
Dynamic Hyperparameter Importance for Efficient Multi-Objective Optimization
- 根据目标权重动态识别关键超参，减少搜索空间
- 在真实任务中提升24%帕累托前沿质量，合成数据上结果翻倍
- 适合需要高效多目标调参的机器学习研究者
选择合适的机器学习模型是一个复杂任务，常需权衡准确率、公平性、能耗等多个目标。现有方法通常将所有超参视为同等重要，忽视了超参重要性（HPI）在不同目标间差异显著的问题。本文提出一种新型动态优化方法，根据优化过程中的目标权衡动态优先处理最具影响力的超参，从而加速收敛。我们首次将超参重要性分析从事后分析转向优化过程中的实时集成，利用近期提出的HyperSHAP方法，结合多目标优化算法ParEGO生成的目标权重，剔除不重要超参，缩小配置空间，使搜索聚焦于关键参数。我们在PyMOO和YAHPO-Gym上的多个任务上进行评估：在超参优化任务中，引入HPI后帕累托前沿质量最高提升24%；在合成数据上，最终结果提升达2倍。
原文摘要 · Abstract (English)
Choosing a suitable ML model is a complex task that can depend on several objectives, e.g., accuracy, fairness, or energy consumption. In practice, this requires trading off multiple, often competing, objectives through multi-objective optimization (MOO). However, existing MOO methods typically treat all hyperparameters as equally important, disregarding that hyperparameter importance (HPI) can vary significantly across objectives. We propose a novel dynamic optimization approach that prioritizes the most influential hyperparameters based on varying objective trade-offs during the search, thereby accelerating empirical convergence. We advance prior work on HPI for MOO from post-analysis to direct, dynamic integration within the optimization, using the recent HPI method HyperSHAP. For this, we leverage the objective weightings naturally produced by the MOO algorithm ParEGO and reduce the configuration space by fixing the unimportant hyperparameters, allowing the search to focus on the important ones. Eventually, we evaluate our method on diverse tasks from PyMOO and YAHPO-Gym. For HPO, integrating HPI yields up to 24% improvement in final Pareto front quality, while on synthetic data, integrating HPI achieves 2x better final results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。