让智能体根据任务情境动态评估能力,避免错误指派。
CADMAS-CTX: Contextual Capability Calibration for Multi-Agent Delegation

- 用上下文相关的后验分布替代固定技能评分。
- 在GAIA上准确率提升至0.442,优于基线0.381。
- 适合需要灵活协作的多智能体系统研究者。
我们重新审视多智能体指派问题,在更现实的假设下:智能体的能力并非固定于某一技能水平,而是依赖任务上下文。例如,一个编码智能体在短任务中表现优异,但在长周期调试中失败;一个规划智能体在简单任务中高效,但在链式依赖任务中退化。静态技能评分会平均化不同情境,导致系统性误指派。为此,我们提出CADMAS-CTX框架,为每个智能体、技能和粗粒度上下文桶维护一个贝塔后验分布,捕捉该任务区域的稳定经验。指派通过风险感知得分完成,结合后验均值与不确定性惩罚,仅当同行明显更优且证据充分时才进行指派。本文贡献三方面:第一,构建分层上下文能力模型,以条件后验替代静态置信度;第二,基于上下文老虎机理论,形式证明在足够上下文异质性下,上下文感知路由比静态路由累积损失更低,刻画偏差-方差权衡;第三,在GAIA和SWE-bench基准上实证验证。在使用GPT-4o的GAIA上,CADMAS-CTX准确率达0.442,显著高于静态基线0.381和AutoGen 0.354(95%置信区间不重叠);在SWE-bench Lite上,解决率从22.3%提升至31.4%。消融实验表明,不确定性惩罚能增强对上下文标签噪声的鲁棒性。结果表明,上下文校准与风险感知指派显著优于静态全局技能分配。
原文摘要 · Abstract (English)
We revisit multi-agent delegation under a stronger and more realistic assumption: an agent's capability is not fixed at the skill level, but depends on task context. A coding agent may excel at short standalone edits yet fail on long-horizon debugging; a planner may perform well on shallow tasks yet degrade on chained dependencies. Static skill-level capability profiles therefore average over heterogeneous situations and can induce systematic misdelegation. We propose CADMAS-CTX, a framework for contextual capability calibration. For each agent, skill, and coarse context bucket, CADMAS-CTX maintains a Beta posterior that captures stable experience in that part of the task space. Delegation is then made by a risk-aware score that combines the posterior mean with an uncertainty penalty, so that agents delegate only when a peer appears better and that assessment is sufficiently well supported by evidence. This paper makes three contributions. First, a hierarchical contextual capability profile replaces static skill-level confidence with context-conditioned posteriors. Second, based on contextual bandit theory, we formally prove context-aware routing achieves lower cumulative regret than static routing under sufficient context heterogeneity, formalizing the bias-variance tradeoff. Third, we empirically validate our method on GAIA and SWE-bench benchmarks. On GAIA with GPT-4o agents, CADMAS-CTX achieves 0.442 accuracy, outperforming static baseline 0.381 and AutoGen 0.354 with non-overlapping 95% confidence intervals. On SWE-bench Lite, it improves resolve rate from 22.3% to 31.4%. Ablations show the uncertainty penalty improves robustness against context tagging noise. Our results demonstrate contextual calibration and risk-aware delegation significantly improve multi-agent teamwork compared with static global skill assignments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。