arXiv:2604.26733cs.AIcs.LG2026-04被引 2

用真实世界结果奖励预测模型,让AI持续学习改进。

FutureWorld: A Live Reinforcement Learning Environment for Predictive Agents with Real-World Outcome Rewards

  • 延迟反馈机制:先预测再补奖,闭环训练预测能力
  • 三类开源代理在多轮训练后准确率与校准度持续提升
  • 适合研究智能体长期学习与真实世界反馈的场景

实时未来预测指在真实事件发生前进行预判,日益依赖基于大语言模型的智能体系统。为构建能持续从现实世界学习的智能体,我们提出FutureWorld——一个可实时运行的智能体强化学习环境,实现预测、结果落地与参数更新的闭环。我们改进并扩展了verl-tool,形成verl-tool-future框架:不同于传统即时奖励,该框架先存储预测期轨迹,待真实结果出现后回填奖励,再重播完整轨迹以更新策略。在三个开源代理上,多轮训练均带来预测准确率、概率评分和校准度的持续提升,证明延迟的真实世界结果反馈可作为有效的强化学习信号。

原文摘要 · Abstract (English)

Live future prediction refers to the task of making predictions about real-world events before they unfold. This task is increasingly studied using large language model-based agent systems, and it is important for building agents that can continually learn from the real world. It can provide a large number of prediction questions grounded in diverse real-world events, while preventing answer leakage. To leverage the advantages of future prediction, we present FutureWorld, a live agentic reinforcement learning environment that closes the training loop between prediction, outcome realization, and parameter updates. Specifically, we modify and extend verl-tool, resulting in a new framework that we call verl-tool-future. Unlike standard reinforcement learning training frameworks that rely on immediate rewards, verl-tool-future stores prediction-time rollouts, backfills rewards after real-world outcomes become available, and then replays the completed trajectories for policy update. Across three open-source agents, successive FutureWorld training rounds lead to consistent improvements in prediction accuracy, probabilistic scoring, and calibration, demonstrating that delayed real-world outcome feedback can serve as an effective reinforcement learning signal.

强化学习未来预测智能体延迟奖励

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。