用轻量学生模型模仿专家规划器，实现低延迟大数据查询。
Agentic Cost-Aware Query Planning with Knowledge Distillation for Big Data Analytics
- 用带约束的强化探索和成本预测优化查询计划生成
- 比默认规划器降低23%延迟，94%满足资源约束
- 学生模型推理快15倍，适合资源受限设备
大数据分析中的查询优化仍计算开销大，尤其在内存和延迟受限环境下，传统优化器难以满足需求。本文提出一种智能查询规划系统，结合基于规则的教师规划器、UCB1贝叶斯探索、成本感知预测与知识蒸馏，将复杂决策压缩至轻量学生规划器。教师规划器采用六种核心优化策略生成SQL计划，UCB1在显式资源约束下高效探索计划空间；随机森林成本模型基于计划特征预测查询延迟，支持成本敏感决策；学生规划器（逻辑回归或梯度提升）通过知识蒸馏学习教师-贝叶斯决策，实现快速推理。在NYC Taxi和IMDB数据集上的评估显示，相比默认规划器，延迟降低23%，资源约束满足率达94%；学生规划器复现最优计划准确率达89%，推理速度提升15倍。单文件实现支持资源受限设备上可复现的大数据分析，代码已开源于https://github.com/mahdinaser/agentic-kd-planner。
原文摘要 · Abstract (English)
Query optimization in big data analytics remains computationally expensive, particularly for resource-constrained environments where traditional optimizers fail to satisfy memory and latency constraints. We present an agentic query planning system that combines a rule-based teacher planner, UCB1 bandit exploration, cost-aware prediction, and knowledge distillation to a lightweight student planner. Our teacher planner generates SQL plans using six key optimization strategies, while UCB1 bandit search efficiently explores the plan space under explicit resource constraints. A Random Forest cost model predicts query latency from plan features, enabling cost-aware decisions. A distilled student planner (Logistic Regression or Gradient Boosting) learns to mimic teacher-bandit decisions for fast inference. Evaluation on NYC Taxi and IMDB datasets demonstrates 23% latency reduction compared to default planners while maintaining 94% constraint satisfaction. The student planner achieves 89% accuracy in replicating optimal plans with 15x faster inference time. Our single-file implementation enables reproducible big-data analytics on resource-limited machines and is publicly available at https://github.com/mahdinaser/agentic-kd-planner.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。