arXiv:2411.15230cs.AIcs.HC2024-11AAAI被引 15

人类与AI协作无法无代价实现优势互补,必须有明确策略。

A No Free Lunch Theorem for Human-AI Collaboration

  • 设计确定性协作规则时,若不始终依赖单一模型,可能比最差个体还差。
  • 只有当一方能识别另一方的明显错误时,协作才可能带来性能提升。
  • 适用于追求可保证性能的人工智能协作场景,如医疗诊断、金融风控。

在二分类任务中,人类与AI协作的黄金标准是互补性——联合表现优于各自单独表现。针对能够输出校准概率预测的两个或多个代理,我们证明了一个类似“无免费午餐”的结果:任何确定性的协作策略(将校准概率映射为二分类决策的函数),若不始终依赖同一代理,则在某些情况下表现会劣于最不准确的个体。这意味着互补性无法“免费”获得。该结果也揭示了一种具有保证的协作模式:一个代理可识别另一个代理的明显错误。此外,该结论可用于分析其他协作技术的成功条件,为人类-AI协作提供指导。

原文摘要 · Abstract (English)

The gold standard in human-AI collaboration is complementarity -- when combined performance exceeds both the human and algorithm alone. We investigate this challenge in binary classification settings where the goal is to maximize 0-1 accuracy. Given two or more agents who can make calibrated probabilistic predictions, we show a "No Free Lunch"-style result. Any deterministic collaboration strategy (a function mapping calibrated probabilities into binary classifications) that does not essentially always defer to the same agent will sometimes perform worse than the least accurate agent. In other words, complementarity cannot be achieved "for free." The result does suggest one model of collaboration with guarantees, where one agent identifies "obvious" errors of the other agent. We also use the result to understand the necessary conditions enabling the success of other collaboration techniques, providing guidance to human-AI collaboration.

人机协作互补性概率校准可靠性保障

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。