arXiv:2606.29654cs.AIcs.MA2026-06

让大模型辩论时自动判断何时执行、何时交给人类,安全可控。

Budgeted Act-or-Defer Multi-Agent LLM Deliberation with Local Reliability Bounds

  • 基于局部置信度边界,动态决定是否执行或转交人类。
  • 在六项基准测试中,误操作率低于16%,自动化率达84%。
  • 无需事后调参,部署前即可确定安全决策点,适合高可靠性场景。

多智能体大模型辩论可提升推理能力,但部署需判断当前答案是否足够可靠以执行,或应升级至人工审核。本文将此问题建模为带预算的执行-延迟决策。每轮中,系统将辩论历史映射为低维状态,利用校准数据计算该状态条件下的k近邻置信下界,并仅当该下界超过用户设定的可靠性阈值时才执行。该证书通过分解β=δ+α+εₐ꜀,分离校准失效、残余动作风险与表征差距。保证为条件性而非分布无关:依赖于有效的局部偏差包络和动作区域表征差距上界,且每个假设均配有可验证的诊断方法。因相同绝对误操作预算在不同难度任务中意义不同,本文将预算相对于训练数据中最终轮次的误差设置,评估安全性采用归一化预算使用率WA/β。在六个基准测试中对比九种基线,该方法在激活数据集上仅使用9–12%预设预算,实现最高84%自动化率与96%执行准确率;在压力测试数据集上则选择延迟而非强制执行不可靠结果。该方法不依赖每任务事后阈值搜索,而是前瞻性地将用户声明的错误动作预算转化为部署前可审计的执行-延迟操作点,前提明确说明。

原文摘要 · Abstract (English)

Multi-agent deliberation among LLMs can improve reasoning, but deployment requires deciding when the current answer is reliable enough to act on and when it should be escalated to human review. We formulate this as budgeted act-or-defer decision making. At each round, the system maps the debate prefix to a low-dimensional state, computes a $k$-nearest-neighbor lower confidence bound on state-conditional correctness using calibration data, and acts only when the bound exceeds a user-specified reliability threshold. The certificate controls wrong actions through the decomposition $β= δ+ α+ \varepsilon_{\mathrm{act}}$, separating calibration failure, residual action risk, and representation gap. The guarantee is conditional, not distribution-free: it relies on a valid local bias envelope and an action-region representation-gap bound, and each assumption is paired with falsification-style diagnostics. Because the same absolute wrong-action budget has different meanings across tasks of different difficulty, we set budgets relative to each task's final-round error using training data only, and evaluate safety by normalized budget usage $\mathrm{WA}/β$. On six benchmarks against nine baselines, the method uses 9--12% of the pre-declared budget on activated datasets, reaching up to 84% automation and 96% acted-on accuracy; on stress-test datasets, it defers rather than forcing unreliable automation. Rather than relying on per-task post-hoc threshold search, the method prospectively converts a user-declared wrong-action budget into an auditable act-or-defer operating point before deployment, under explicitly stated assumptions.

多智能体大模型推理安全决策自动化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。