arXiv:2606.23983cs.CRcs.AI2026-06

用四类组合策略让不可靠模型变可靠,省钱又高效。

Maestro Order: A Model-Agnostic Orchestration Harness

论文配图:Maestro Order: A Model-Agnostic Orchestration Harness
图 1 · 摘自论文原文
  • 用分解、集成、验证、递归四类结构组合模型,动态分配算力。
  • 验证可使可靠性从55%提升至99.9%,投票效果受共性错误限制。
  • 预算感知控制器让目标可靠性成本降低至纯投票的极小部分。

单次前向传播的强模型虽快速流畅,却常出错(如语言模型中的幻觉)——足够正确可用,又足够错误危险。本文提出 Maestro Order,一种与模型无关的编排框架,通过四种结构原语(分解、集成、验证、递归)和预算感知控制器,将不可靠求解器转化为可靠系统。该框架将任意模型视为黑盒求解器,统一接口接入,叠加在线评估的验证器集合,并根据每单位成本的边际可靠性,动态分配验证与投票资源。我们给出架构、消息与状态设计、控制器算法及工程实现,确保系统确定性、可观测性与容错性。随后定义评估方法(固定成本下的可靠性、覆盖率、校准性与消融实验),并通过参数化求解器/验证器模型的蒙特卡洛仿真验证结果。仿真定量复现了理论规律:验证可几何级提升可靠性(如0.55→0.98,四重门达0.999);投票仅在超随机水平有效,且受限于共性错误;预算感知控制器在各阶段选择最廉价机制,以极小成本达成目标可靠性。最后讨论失败模式(验证器作弊、相关错误、分解误差累积),并提出建议:构建稳健检查器,多样化求解器,让控制器把算力投向信息丰富处。

原文摘要 · Abstract (English)

A single forward pass of a capable model is a fast, fluent, and unreliable problem-solver: it is right often enough to be useful and wrong often enough to be dangerous; in language models, such confident errors are known as hallucinations. We present Maestro Order, a model-agnostic orchestration harness that turns unreliable solvers into reliable problem-solving systems by composing them according to four structural primitives (decompose, ensemble, verify, and recurse) and a budget-aware controller that decides where to spend compute. The harness treats any model as a black-box base solver behind a uniform interface, layers a verifier ensemble whose discrimination is measured online, and allocates verification and voting to the stages with the highest marginal reliability per unit cost. We give the architecture, the message and state schema, the controller algorithm, and the engineering that makes it deterministic, observable, and fault-tolerant. We then specify an evaluation methodology (reliability at fixed cost, coverage, calibration, and ablations) and report results from a faithful Monte Carlo simulation of the harness over a parameterized solver/verifier model. The simulation reproduces the predicted laws quantitatively: verification amplifies reliability geometrically (e.g. $0.55\to0.98$ with two gates, $\to0.999$ with four), voting helps only above chance and is limited by shared errors, and a budget-aware controller reaches a target reliability at a small fraction of the cost of voting alone by selecting the cheapest mechanism for each regime. We close with failure modes (verifier gaming, correlated errors, and decomposition error compounding) and concrete guidance: build robust checkers, diversify solvers, and let the controller put compute where the information is.

模型编排可靠性算力优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。