让模型同时参考最可靠的k个专家,提升决策准确性与成本效益。
Why Ask One When You Can Ask $k$? Learning-to-Defer to the Top-$k$ Experts
- 提出Top-k学习性拒答框架,可同时调用多个专家协作决策。
- 在多种场景下,相比单专家方案,准确率提升且成本更低。
- 自适应调整每项查询的专家数量,适合复杂任务和动态成本环境。
现有学习性拒答(L2D)框架仅支持单专家拒答,迫使每个查询只能依赖单一专家,限制了集体智慧的应用。本文首次提出Top-k学习性拒答框架,将查询分配给前k个最具成本效益的实体。该方法统一并严格推广了先前方法,包括单阶段与双阶段设置、选择性预测及经典级联结构。特别地,它将传统的Top-1拒答规则作为特例,并在k>1时实现有原则的多专家协作。进一步提出自适应变体Top-k(x) L2D,根据输入难度、专家质量与咨询成本动态学习最优专家数量。为支持实际训练,设计了一种贝叶斯一致、单阶段下H_h一致、双阶段下(H_r,H_g)一致的新代理损失。关键优势在于该损失不依赖k,只需训练一次即可灵活应用于不同k值。跨两种设置的实验表明,Top-k与Top-k(x)均显著改善准确率-成本权衡,开辟了多专家拒答的新方向。
原文摘要 · Abstract (English)
Existing Learning-to-Defer (L2D) frameworks are limited to single-expert deferral, forcing each query to rely on only one expert and preventing the use of collective expertise. We introduce the first framework for Top-$k$ Learning-to-Defer, which allocates queries to the $k$ most cost-effective entities. Our formulation unifies and strictly generalizes prior approaches, including the one-stage and two-stage regimes, selective prediction, and classical cascades. In particular, it recovers the usual Top-1 deferral rule as a special case while enabling principled collaboration with multiple experts when $k>1$. We further propose Top-$k(x)$ Learning-to-Defer, an adaptive variant that learns the optimal number of experts per query based on input difficulty, expert quality, and consultation cost. To enable practical learning, we develop a novel surrogate loss that is Bayes-consistent, $\mathcal{H}_h$-consistent in the one-stage setting, and $(\mathcal{H}_r,\mathcal{H}_g)$-consistent in the two-stage setting. Crucially, this surrogate is independent of $k$, allowing a single policy to be learned once and deployed flexibly across $k$. Experiments across both regimes show that Top-$k$ and Top-$k(x)$ deliver superior accuracy-cost trade-offs, opening a new direction for multi-expert deferral in L2D.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。