用数据库优化思路，让多个大模型协作答题更省资源、更准。
Opti-Q: A Constraint-Based Optimization Framework for Multi-LLM Question Planning
- 把大模型调用当操作符，构建执行图找最优组合
- 在预算限制下，答题质量比基线高58%和41%
- 适合需要多模型协作且资源受限的问答系统
尽管大语言模型（LLMs）能实现强问答能力，但在预算约束下部署仍受非确定性及异构资源特征（成本、延迟、能耗）影响。我们提出OPTI-Q，一种受数据库启发的成本优化框架，采用先规划后执行的范式实现多模型协同。将LLM调用建模为执行有向无环图中的物理操作符，针对每个问题搜索在用户指定资源约束下，兼顾答案质量（QoA）与财务成本、延迟、能耗的最优方案。计划可包含串行操作符传递中间答案作为上下文，或并行/融合操作符同时运行模型并合并输出。为避免执行每种候选计划，OPTI-Q使用PERFDB——一个从基准测试和执行轨迹中动态更新的统计目录——估算单个操作符及组合子计划的答案质量和资源开销。基于这些估计，进行帕累托前沿搜索，并根据用户偏好选择最终计划。在MMLU-Pro和SimpleQA数据集上，用户指定预算下，OPTI-Q相比基线平均答案质量提升约58%和41%，证明数据库式规划能带来更好的质量-资源权衡，适用于多模型问答场景。
原文摘要 · Abstract (English)
While large language models (LLMs) enable strong question answering (QA), budgeted deployment is complicated by nondeterminism and heterogeneous resource profiles (cost, latency, and energy). We present OPTI-Q, a database-inspired, cost-based optimizer that implements a plan-before-execute paradigm for multi-LLM orchestration. OPTI-Q models LLM invocations as physical operators in an execution DAG and, for each question, searches for plans that optimize answer quality (QoA) while trading off financial cost, latency, and energy under user-specified resource constraints. Plans can include sequential operators that pass intermediate answers as context and parallel/blend operators that run models concurrently and merge their outputs. To search this space without executing each candidate plan, OPTI-Q uses PERFDB, a statistics catalog populated and refreshed from benchmarks and execution traces, to estimate the QoA and resource costs of both individual operators and composed subplans. Using these estimates, OPTI-Q performs Pareto-frontier search and selects a final plan based on user preferences. On MMLU-Pro and SimpleQA under user-specified budgets, OPTI-Q improves average QoA by ~58% and ~41% over baselines at comparable cost, demonstrating that database-style planning yields better quality-resource trade-offs for multi-LLM QA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。