多任务贝叶斯优化中常用方法会误估任务相关性，本文提出三种改进方案。
Pitfalls and Remedies for Multi-Task Bayesian Optimization
- 将每任务均值和尺度设为模型参数，避免标准化引入误差
- 限制任务协方差为非负相关，提升估计稳定性
- 部分共用源与目标设计点，增强信息传递效率
贝叶斯优化常通过相关源任务的数据来热启动目标任务，多任务高斯过程是标准代理模型。我们在受控环境下重新审视这一默认做法，发现即使在最简单的仿射相关情形下，该方法仍会错误估计跨任务相关性，而此时理想的迁移学习应能成功。我们追溯失败原因在于两个独立的结构机制：每任务标准化虽解决仿射切片模糊性，却将有限样本对齐误差传播至相关性恢复；边际似然仅以逐样本速率识别相关性，而无重叠设计的高斯过程进一步稀释了该信号。基于分析，我们提出三种保守修正策略：将每任务均值与尺度设为模型参数、限制任务协方差为非负相关、部分共用源与目标设计点。在合成多任务问题及基于代理的超参调优迁移任务中，这些方法在简单实例上恢复了仅用目标数据的基准性能，但在更复杂实例以及大多数基于秩或潜在上下文的变体中，整体失效仍持续存在。
原文摘要 · Abstract (English)
Bayesian optimization routinely warm-starts a target experiment with data from related source tasks, and the multi-task Gaussian process is the textbook surrogate for the job. We revisit this default in a controlled setting and find that it misestimates the cross-task correlation even in the simplest non-trivial case, affinely related source and target tasks, where a working transfer learning method should obviously succeed. We trace the failure to two independent structural mechanisms. Per-task standardization, the textbook fix for the affine slice ambiguity, propagates a finite-sample alignment error into the recovered correlation. The marginal likelihood itself identifies the correlation only at a per-sample rate that a Gaussian process at non-overlapping designs further dilutes. We propose three conservative remedies that follow from the analysis: promoting per-task means and scales to model parameters, restricting the task covariance to non-negative correlations, and co-locating part of the source and target designs. Across synthetic multi-task problems and surrogate-based hyperparameter tuning transfer, these remedies recover the target-only baseline on the simple instances, while the broader failure persists on harder instances and across most rank-based and latent-context variants.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。