Optuna的约束超参优化方法本质是联合概率建模，比独立假设更稳健。
Optuna Constrained Tree-Structured Parzen Estimator Is a Joint Density Generalization of c-TPE
- 用联合概率分布统一建模目标与约束，替代独立假设
- 联合方法对重复约束不敏感，独立方法会因重复因子退化
- 适合需要稳定约束处理的工业级超参调优场景
约束超参数优化在实践中很常见，但Optuna广泛使用的约束TPE缺乏算法分析。尽管c-TPE采用目标与约束独立的期望约束改进（ECI）策略，Optuna却使用单一联合密度对两者建模。本文证明Optuna的约束TPE即为联合c-TPE——采用联合似然的相同ECI采集函数。我们证明联合c-TPE在约束重复时保持不变，而独立c-TPE会随重复因子累积而性能下降。文中还讨论了两种形式的实际权衡及未来研究方向。
原文摘要 · Abstract (English)
Constrained hyperparameter optimization (HPO) is common in practice, yet Optuna's widely used constrained TPE lacks algorithmic analysis. While c-TPE proposes an expected constrained improvement (ECI) approach assuming independence between the objective and constraints, Optuna uses a single joint density over both. We show that Optuna's constrained TPE is joint c-TPE -- the same ECI acquisition function using a joint likelihood. We demonstrate joint c-TPE is invariant to constraint duplication whereas independent c-TPE degrades as the product accumulates duplicated factors. We outline practical tradeoffs between the formulations and directions for future study.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。