让机器与人互动协商,提升预测准确率。
Tractable Agreement Protocols
- 设计交互协议,让模型与人类轮流反馈并更新判断。
- 无论多少可能结果,达成共识的轮次不随数量增长。
- 适合需要人机协同决策的场景,如医疗诊断。
我们提出一种高效转换方法,将任意机器学习算法变为交互协议,使模型与另一方(如人类)协作达成预测共识并提升准确率。该方法对各方施加可计算、可统计处理的校准条件,是贝叶斯理性的合理放松,即使在无先验设定下也合理,显著推广了奥曼的经典“共识定理”。在协议中,模型先给出预测,人类选择同意或提供反馈;模型据此更新并修正预测,人类也可调整信念。此过程持续至双方达成一致。初始设定扩展奥曼定理,双方就一维期望值迭代共享估计,在弱于Aaronson'05的假设下仍能收敛。随后考虑持有d个结果分布信念的情况,探索两种反馈机制:一是向量化预测估计,二是基于决策理论——人类根据效用选择最优动作并传递。在此设定中,达成共识的轮次数独立于d。最后推广至多于两方情形,计算复杂度随参与者数线性增长。协议采用简单高效条件,所得预测优于任一方单独表现。
原文摘要 · Abstract (English)
We present an efficient reduction that converts any machine learning algorithm into an interactive protocol, enabling collaboration with another party (e.g., a human) to achieve consensus on predictions and improve accuracy. This approach imposes calibration conditions on each party, which are computationally and statistically tractable relaxations of Bayesian rationality. These conditions are sensible even in prior-free settings, representing a significant generalization of Aumann's classic "agreement theorem." In our protocol, the model first provides a prediction. The human then responds by either agreeing or offering feedback. The model updates its state and revises its prediction, while the human may adjust their beliefs. This iterative process continues until the two parties reach agreement. Initially, we study a setting that extends Aumann's Agreement Theorem, where parties aim to agree on a one-dimensional expectation by iteratively sharing their current estimates. Here, we recover the convergence theorem of Aaronson'05 under weaker assumptions. We then address the case where parties hold beliefs over distributions with d outcomes, exploring two feedback mechanisms. The first involves vector-valued estimates of predictions, while the second adopts a decision-theoretic approach: the human, needing to take an action from a finite set based on utility, communicates their utility-maximizing action at each round. In this setup, the number of rounds until agreement remains independent of d. Finally, we generalize to scenarios with more than two parties, where computational complexity scales linearly with the number of participants. Our protocols rely on simple, efficient conditions and produce predictions that surpass the accuracy of any individual party's alone.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。