大规模训练语言模型提升事件预测能力,解决噪声、知识滞后等难题。
Advancing Event Forecasting through Massive Training of Large Language Models: Challenges, Solutions, and Broader Impacts
- 用贝叶斯网络和反事实事件缓解训练噪声与知识断层问题。
- 引入辅助奖励信号,使模型在复杂预测中表现接近顶尖人类专家。
- 适合关注AI预测、认知增强与社会智能的科研人员参考。
近期研究显示,先进语言模型正逐步达到超预测者水平的事件预测能力,强化学习也能提升未来预测表现。基于此趋势,本文主张应大力推动超预测级事件预测大模型的规模化训练。针对训练中的三大挑战——数据噪声稀疏性、知识截止问题及奖励结构简单化,提出解决方案:利用假设事件贝叶斯网络、挖掘低记忆度与反事实事件、设计辅助奖励信号。在数据层面,建议激进使用市场数据、公开数据与爬取数据以支持大规模训练与评估。这些技术进步有望让AI在更广泛领域为社会提供预测智能。本文系统梳理了实现超预测者级AI的关键路径,旨在激发研究者对这一方向的关注。
原文摘要 · Abstract (English)
Many recent papers have studied the development of superforecaster-level event forecasting LLMs. While methodological problems with early studies cast doubt on the use of LLMs for event forecasting, recent studies with improved evaluation methods have shown that state-of-the-art LLMs are gradually reaching superforecaster-level performance, and reinforcement learning has also been reported to improve future forecasting. Additionally, the unprecedented success of recent reasoning models and Deep Research-style models suggests that technology capable of greatly improving forecasting performance has been developed. Therefore, based on these positive recent trends, we argue that the time is ripe for research on large-scale training of superforecaster-level event forecasting LLMs. We discuss two key research directions: training methods and data acquisition. For training, we first introduce three difficulties of LLM-based event forecasting training: noisiness-sparsity, knowledge cut-off, and simple reward structure problems. Then, we present related ideas to mitigate these problems: hypothetical event Bayesian networks, utilizing poorly-recalled and counterfactual events, and auxiliary reward signals. For data, we propose aggressive use of market, public, and crawling datasets to enable large-scale training and evaluation. Finally, we explain how these technical advances could enable AI to provide predictive intelligence to society in broader areas. This position paper presents promising specific paths and considerations for getting closer to superforecaster-level AI technology, aiming to call for researchers' interest in these directions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。