通过学习资源估算,实现阿里云AnalyticDB的智能弹性扩容,大幅降低查询成本。
ScaleSense: Cost-Intelligent Scaling Framework via Learned Resource Estimation in Alibaba AnalyticDB

- 基于查询计划与硬件特征的多维度编码器,预估资源需求
- 量化预测覆盖率达标,资源配置准确率提升76.7%
- 无需重训练即可适配不同业务目标,适合高并发数据仓库场景
云原生无服务器数据仓库通过计算与存储解耦实现细粒度弹性,但针对高度异构的即席查询,最优资源配置仍是工业界难题。对阿里云AnalyticDB生产负载的分析揭示了一个高成本的‘配置陷阱’:用户因惧怕资源耗尽而盲目超配,浪费巨额预算却无法缓解非CPU瓶颈(如I/O饱和)。为此,我们提出ScaleSense,一种主动的、基于查询级别的资源动态扩容框架。其核心为多面查询编码器,联合建模查询计划拓扑与硬件规格;关键在于基于分位数的资源预测器,可估计多维物理资源开销,作为安全可靠的扩容依据。自动扩容控制器在性能-成本帕累托前沿中导航,根据业务优先级动态调整分配,无需模型重训练。在超过136万条生产查询上的评估显示,ScaleSense达到业界最优预测精度,预测区间覆盖率良好。相比最佳基线,资源配置选择准确率提升76.7%。在性能优化策略下,满足用户性能要求的同时,最高可将成本降低5.22倍,证实其在生产环境中的高效实用性。
原文摘要 · Abstract (English)
Cloud-native serverless data warehouses achieve fine-grained elasticity by decoupling storage from compute, yet determining the optimal resource allocation for highly heterogeneous ad-hoc queries remains a formidable industrial challenge. Our analysis of production workloads in Alibaba AnalyticDB exposes a costly ``provisioning trap'': the fear of catastrophic resource depletion drives users to blindly over-provision resources, wasting immense monetary budgets without alleviating non-CPU bottlenecks (e.g., I/O saturation). To break this impasse, we propose ScaleSense, a proactive, query-level resource scaling framework. Specifically, it features a multi-faceted query encoder that jointly models plan topologies and hardware specifications. Crucially, a quantile-based resource predictor estimates multi-dimensional physical footprints, acting as a reliable safety net for optimal resource scaling. An auto-scaling controller then navigates the performance-cost Pareto frontier, dynamically tailoring allocations to specific business priorities without requiring model retraining. Evaluations on over 1.36 million production queries show that ScaleSense achieves state-of-the-art prediction accuracy with good prediction interval coverage. By achieving a 76.7% relative improvement in optimal resource configuration selection over the best baseline, this approach addresses the critical performance-cost trade-off while maintaining low-overhead inference latency, confirming its practical performance in production deployments. Under the performance-optimization policy, ScaleSense satisfies user-defined performance requirements while reducing monetary cost by up to 5.22x.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。