混合预测系统提升地缘事件预判准确率,人机协作更胜纯人工。
Hybrid Forecasting of Geopolitical Events
- 融合人类与机器预测,通过基准锚定和技能加权聚合。
- 8个月1085人预测398个真实问题,混合模型显著优于纯人类基线。
- 适合需要大规模、高精度预测的决策支持场景。
可靠的决策依赖于对军事冲突、疾病暴发等实际结果的准确预测。为提升众包预测的准确性,我们开发了SAGE混合预测系统,整合人类与机器生成的预测。该系统提供交互平台,使用户可参考机器模型作为客观基准,并在聚合时根据贴近度和评估出的技能水平对人机预测进行加权,同时校正过度自信。在规模超过同类竞赛的混合预测竞赛(HFC)中,1085名用户历时八个月对398个真实世界问题进行预测。主要结果表明,相较于无机器预测的人类基线,混合系统生成的预测更为准确;具备机器预测信息的熟练预测者表现优于仅见历史数据者。此外,将机器预测纳入聚合算法提升了整体性能,兼具准确性和可扩展性。这表明,此类混合预测系统或可在减少人力投入的同时,维持高水平的预测精度,适用于大规模预测任务。
原文摘要 · Abstract (English)
Sound decision-making relies on accurate prediction for tangible outcomes ranging from military conflict to disease outbreaks. To improve crowdsourced forecasting accuracy, we developed SAGE, a hybrid forecasting system that combines human and machine generated forecasts. The system provides a platform where users can interact with machine models and thus anchor their judgments on an objective benchmark. The system also aggregates human and machine forecasts weighting both for propinquity and based on assessed skill while adjusting for overconfidence. We present results from the Hybrid Forecasting Competition (HFC) - larger than comparable forecasting tournaments - including 1085 users forecasting 398 real-world forecasting problems over eight months. Our main result is that the hybrid system generated more accurate forecasts compared to a human-only baseline which had no machine generated predictions. We found that skilled forecasters who had access to machine-generated forecasts outperformed those who only viewed historical data. We also demonstrated the inclusion of machine-generated forecasts in our aggregation algorithms improved performance, both in terms of accuracy and scalability. This suggests that hybrid forecasting systems, which potentially require fewer human resources, can be a viable approach for maintaining a competitive level of accuracy over a larger number of forecasting questions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。