通过黑箱在线调优提升大模型服务性能，推动系统指标纳入可信AI事实表。
Improving LLM Performance Through Black-Box Online Tuning: A Case for Adding System Specs to Factsheets for Trusted AI
- 仅用短段落端到端测量，无需内部探针，通过爬山法优化请求吞吐量。
- 提出的新控制器在大模型服务中显著提升满足服务水平目标的请求处理能力。
- 强调将系统性能与可持续性指标加入AI系统事实表，助力可信AI落地。
本文提出一种新型黑箱在线控制器，仅依赖短段落的端到端测量结果，无需内部仪器监测，通过爬山法最大化良好吞吐量（goodput），即满足服务级别目标的请求吞吐量。我们提供了实证证据，证明该设计具有可行性。以大模型服务为例，进一步探讨将系统性能与可持续性指标整合进组织采用的AI系统事实表（Factsheets）的重要性，推动构建更透明、可信赖的AI应用体系。
原文摘要 · Abstract (English)
In this paper, we present a novel black-box online controller that uses only end-to-end measurements over short segments, without internal instrumentation, and hill climbing to maximize goodput, defined as the throughput of requests that satisfy the service-level objective. We provide empirical evidence that this design is well-founded. Using this advance in LLM serving as a concrete example, we then discuss the importance of integrating system performance and sustainability metrics into Factsheets for organizations adopting AI systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。