提出分层代理架构，让大模型按需计算，省下四成算力还能保持高准确率。
Pyramid MoA: A Probabilistic Framework for Cost-Optimized Anytime Inference
- 用决策理论设计路由器，只在必要时调用更深层模型
- 在MBPP上拦截81.6%错误，在GSM8K上节省42.9%算力接近最优表现
- 跨任务迁移能力强，零样本适配新数据集且大幅降本
我们发现大模型级联与路由机制本质上解决了一个随时计算问题——经典人工智能中随计算资源增加逐步优化结果的算法。本文正式建立该关联，提出金字塔型多代理（Pyramid MoA）架构，其由基于决策理论的路由器控制，仅在必要时升级推理层级。我们确立了概率性随时性质，具有可证明的单调性保证，并从计算价值理论推导出广义升级规则，适用于不完美预测器，将Hansen-Zilberstein监控框架拓展至随机大模型推理。在MBPP数据集上，路由器拦截81.6%的错误；在GSM8K/MMLU上，系统近乎达到68.1%的最优基准准确率，同时实现最高达42.9%的计算节省。路由器成功将零样本能力迁移至未见过的基准：在HumanEval（81.1%）和MATH 500（58.0%）上匹配最优基线，且显著降低成本。进一步发现，上下文条件下的锚定效应——正确小模型推理可提升最优基线准确率最高+19.2个百分点，错误推理则下降最多-18.0个百分点，揭示了分层多代理架构中的根本矛盾。
原文摘要 · Abstract (English)
We observe that LLM cascading and routing implicitly solves an anytime computation problem -- a class of algorithms, well-studied in classical AI, that improve solutions as additional computation is allocated. We formalize this connection and propose Pyramid MoA, a hierarchical Mixture-of-Agents architecture governed by a decision-theoretic router that escalates queries only when necessary. We establish a Probabilistic Anytime Property with provable monotonicity guarantees and derive a generalized escalation rule from Value of Computation theory that accounts for imperfect oracles, extending the Hansen-Zilberstein monitoring framework to stochastic LLM inference. On MBPP, the router intercepts 81.6% of bugs; on GSM8K/MMLU, the system nearly matches the 68.1% Oracle baseline while achieving up to 42.9% compute savings. The router transfers zero-shot to unseen benchmarks: matching Oracle accuracy on HumanEval (81.1%) and MATH 500 (58.0%) with significant cost reductions. We further discover a context-conditioned anchoring effect across four benchmarks: passing correct SLM reasoning improves Oracle accuracy by up to +19.2pp, while incorrect reasoning degrades it by up to -18.0pp, revealing a fundamental tension in hierarchical MoA architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。