arXiv:2411.04525cs.DBcs.AI2024-11被引 7

GenJoin用生成式方法学习子计划提示,显著优于传统优化器。

GenJoin: Conditional Generative Plan-to-Plan Query Optimizer that Learns from Subplan Hints

  • 将查询优化视为生成任务,从随机子计划提示中学习
  • 在两个真实基准上均显著超越PostgreSQL和现有方法
  • 适合需要高效稳定优化的数据库系统研发人员

查询优化正面临经典算法被机器学习算法挑战的局面。当前趋势表明,更优策略是借助数十年数据库研究积累,通过指定连接类型、扫描方式或连接顺序等提示来缩小计划搜索空间,而非完全用机器学习模型替代经典优化器。尤其在经典优化器无法穷举所有逻辑与物理计划时,此方法尤为关键。然而,即使协同学习的优化器仍受限于海量训练数据、推理阶段生成缓慢以及在不同工作负载下结果不稳定等问题。本文提出GenJoin——首个将查询优化视为生成任务的新型学习型优化器,能从随机子计划提示中学习,并生成优于经典优化器的查询计划。在两个知名真实世界基准上,经严格机器学习评估,GenJoin在多种工作负载下均显著且一致地超越PostgreSQL及最先进方法。

原文摘要 · Abstract (English)

Query optimization has become a research area where classical algorithms are being challenged by machine learning algorithms. At the same time, recent trends in learned query optimizers have shown that it is prudent to take advantage of decades of database research and augment classical query optimizers by shrinking the plan search space through different types of hints (e.g. by specifying the join type, scan type or the order of joins) rather than completely replacing the classical query optimizer with machine learning models. It is especially relevant for cases when classical optimizers cannot fully enumerate all logical and physical plans and, as an alternative, need to rely on less robust approaches like genetic algorithms. However, even symbiotically learned query optimizers are hampered by the need for vast amounts of training data, slow plan generation during inference and unstable results across various workload conditions. In this paper, we present GenJoin - a novel learned query optimizer that considers the query optimization problem as a generative task and is capable of learning from a random set of subplan hints to produce query plans that outperform the classical optimizer. GenJoin is the first learned query optimizer that significantly and consistently outperforms PostgreSQL as well as state-of-the-art methods on two well-known real-world benchmarks across a variety of workloads using rigorous machine learning evaluations.

查询优化生成模型数据库机器学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。