用AI代理自动调优异构存储系统,性能提升超5倍,延迟降低近90%。
StorageXTuner: An LLM Agent-Driven Automatic Tuning Framework for Heterogeneous Storage Systems
- 分四类智能体协同工作,动态探索配置空间。
- 在多个数据库上实现最高575%吞吐提升,尾延迟降88%。
- 适合需要跨系统自动调优的工程师和系统研发者。
自动配置存储系统难度大:参数空间庞大,且受工作负载、部署环境和版本变化影响。传统启发式与机器学习调优方法通常依赖特定系统、需手动集成,且在环境变化时性能下降。近期基于大模型的方法虽有进展,但多将调优视为单次、系统专属任务,限制了跨系统复用、探索能力与验证强度。本文提出 StorageXTuner,一个面向异构存储引擎的 LLM 代理驱动自动调优框架。该框架通过四个智能体分离职责:执行器(沙箱基准测试)、提取器(性能摘要)、搜索器(基于洞察的配置探索)与反思器(洞察生成与管理)。设计结合洞察驱动的树搜索与分层记忆机制,促进经实证验证的优化见解积累,并使用轻量级检查器防止不安全操作。我们实现了原型,在 RocksDB、LevelDB、CacheLib 与 MySQL InnoDB 上,采用 YCSB、MixGraph 与 TPC-H/C 进行评估。相比默认配置和 ELMo-Tune,StorageXTuner 最高实现 575% 和 111% 的吞吐提升,尾延迟(p99)最多降低 88% 和 56%,且收敛所需尝试次数更少。
原文摘要 · Abstract (English)
Automatically configuring storage systems is hard: parameter spaces are large and conditions vary across workloads, deployments, and versions. Heuristic and ML tuners are often system specific, require manual glue, and degrade under changes. Recent LLM-based approaches help but usually treat tuning as a single-shot, system-specific task, which limits cross-system reuse, constrains exploration, and weakens validation. We present StorageXTuner, an LLM agent-driven auto-tuning framework for heterogeneous storage engines. StorageXTuner separates concerns across four agents - Executor (sandboxed benchmarking), Extractor (performance digest), Searcher (insight-guided configuration exploration), and Reflector (insight generation and management). The design couples an insight-driven tree search with layered memory that promotes empirically validated insights and employs lightweight checkers to guard against unsafe actions. We implement a prototype and evaluate it on RocksDB, LevelDB, CacheLib, and MySQL InnoDB with YCSB, MixGraph, and TPC-H/C. Relative to out-of-the-box settings and to ELMo-Tune, StorageXTuner reaches up to 575% and 111% higher throughput, reduces p99 latency by as much as 88% and 56%, and converges with fewer trials.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。