为超参优化预热设计可捕捉关键样本的表征学习方法
Are encoders able to learn landmarkers for warm-starting of Hyperparameter Optimization?
- 通过度量学习与关键样本重建，让编码器聚焦于关键样本特征
- 表征能有效对齐关键样本，但未带来超参优化性能显著提升
- 适合需要精准初始化的自动化机器学习场景
为元学习目的高效表示异构表格数据仍是未解难题。现有方法依赖通用表征，本文提出两种专为超参优化预热任务设计的新表征学习方法，均满足我们提出的‘捕捉关键样本特性’的要求。第一种基于深度度量学习，第二种基于关键样本重建。通过两种方式评估：一是目标元任务的性能提升，二是所提要求的满足程度。实验表明，所提编码器能有效学习与关键样本对齐的表征，但未必直接转化为超参优化预热任务中的显著性能增益。
原文摘要 · Abstract (English)
Effectively representing heterogeneous tabular datasets for meta-learning purposes is still an open problem. Previous approaches rely on representations that are intended to be universal. This paper proposes two novel methods for tabular representation learning tailored to a specific meta-task - warm-starting Bayesian Hyperparameter Optimization. Both follow the specific requirement formulated by ourselves that enforces representations to capture the properties of landmarkers. The first approach involves deep metric learning, while the second one is based on landmarkers reconstruction. We evaluate the proposed encoders in two ways. Next to the gain in the target meta-task, we also use the degree of fulfillment of the proposed requirement as the evaluation metric. Experiments demonstrate that while the proposed encoders can effectively learn representations aligned with landmarkers, they may not directly translate to significant performance gains in the meta-task of HPO warm-starting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。