通过细粒度预测优化并发查询调度,显著降低数据库响应时间。
Improving DBMS Scheduling Decisions with Fine-grained Performance Prediction on Concurrent Queries -- Extended
- 将数据库视为黑箱,用细粒度预测器估算并发查询的系统运行时间
- 在Postgres上平均缩短16.2%-28.2%端到端延迟,尾部延迟降33.6%-38.9%
- 无需修改内核,可跨引擎部署,适合追求低延迟的生产环境
查询调度直接影响数据库管理系统(DBMS)的查询性能。深度集成调度器需修改内核,定制化程度高但实现周期长;而非侵入式调度器通过控制查询准入和重排执行顺序,在不改动内核的前提下实现快速部署,但多数依赖简化成本模型与启发式规则,难以准确刻画并发场景下查询间的交互及系统状态变化,易导致次优决策。本文提出IconqSched,一种新的原则性非侵入式调度器,旨在优化查询的执行顺序与时机,以提升用户感知的端到端运行时间(包括排队时间与系统执行时间)。其核心是新设计的细粒度预测器Iconq,将DBMS视为黑箱,能精准预测不同系统状态下并发查询的系统运行时间。基于此预测,IconqSched捕捉了不同查询组合与负载下的运行时波动,并采用贪心算法动态决定提交哪些查询及何时提交。在真实工作负载上对比,IconqSched在Postgres上平均减少16.2%-28.2%的端到端延迟,尾部延迟降低33.6%-38.9%;在Redshift上平均降低10.3%-14.1%,尾部延迟降低14.9%-22.2%。
原文摘要 · Abstract (English)
Query scheduling is a critical task that directly impacts query performance in database management systems (DBMS). Deeply integrated schedulers, which require changes to DBMS internals, are usually customized for a specific engine and can take months to implement. In contrast, non-intrusive schedulers make coarse-grained decisions, such as controlling query admission and re-ordering query execution, without requiring modifications to DBMS internals. They require much less engineering effort and can be applied across a wide range of DBMS engines, offering immediate benefits to end users. However, most existing non-intrusive scheduling systems rely on simplified cost models and heuristics that cannot accurately model query interactions under concurrency and different system states, possibly leading to suboptimal scheduling decisions. This work introduces IconqSched, a new, principled non-intrusive scheduler that optimizes the execution order and timing of queries to enhance total end-to-end runtime as experienced by the user query queuing time plus system runtime. Unlike previous approaches, IconqSched features a novel fine-grained predictor, Iconq, which treats the DBMS as a black box and accurately estimates the system runtime of concurrently executed queries under different system states. Using these predictions, IconqSched is able to capture system runtime variations across different query mixes and system loads. It then employs a greedy scheduling algorithm to effectively determine which queries to submit and when to submit them. We compare IconqSched to other schedulers in terms of end-to-end runtime using real workload traces. On Postgres, IconqSched reduces end-to-end runtime by 16.2%-28.2% on average and 33.6%-38.9% in the tail. Similarly, on Redshift, it reduces end-to-end runtime by 10.3%-14.1% on average and 14.9%-22.2% in the tail.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。