arXiv:2607.18084cs.AIcs.CL2026-07被引 1

用世界杯动态评测语言模型的足球预测能力,更细粒度判断表现优劣。

WorldCupArena: Fine-Grained Evaluation of Language Models and Deep-Research Agents on Football Forecasting

  • 模型可选接收信息或自主搜索,预测比分、球员、赛事结果等多维度内容。
  • 最佳模型在得分线评分上优于博彩市场和人类球迷,但整体准确率差距小。
  • 支持未来赛事扩展,适合评估大模型在动态决策中的真实表现。

赛前预测足球比赛不仅需历史数据,还需实时信息与明确判断。我们提出WorldCupArena,一个针对语言模型与深度研究代理的动态基准,首次应用于2026年FIFA世界杯,未来可复用于其他联赛。每场比赛前,模型可获取通用证据包或自主检索信息,预测比赛结果、比分、关键球员与事件、比赛统计及赛事最终结果。赛后将预测与实际结果比对,评估结果准确率、精确比分准确率,以及允许小幅偏差的得分线评分,并涵盖其他预测任务得分。不同系统在结果准确率相近时,细节预测差异显著;四个系统均预测西班牙夺冠,其中两个还原了决赛对阵。相比博彩市场与人类球迷基线,最佳系统在结果与精确比分准确率上仅微弱提升,但在得分线评分上表现更优。新赛程可随时加入,无需使用已知结果,确保评估时效性。代码、预测与评估脚本将公开发布。

原文摘要 · Abstract (English)

Predicting a football match before kickoff requires more than knowing past results: a model must use changing information and make a clear prediction before the answer is available. We present WorldCupArena, a dynamic benchmark for language models and deep-research agents. The 2026 FIFA World Cup is its first evaluation, and the same process can be reused for future leagues and cups. Before each match, a model either receives a common evidence package or searches for information itself. It predicts the result and score, likely players and events, match statistics, and the outcome of the competition. After the match, these predictions are compared with the recorded result. We report result accuracy, exact-score accuracy, and a scoreline score that gives some credit when a predicted score is close but not exact, together with scores for the other prediction tasks. Across systems, similar result accuracy can mask larger differences in detailed predictions. Four systems predicted champion Spain, and two of them also recovered the exact final pairing. Compared with betting-market and human-fan baselines, the best system shows only small gains in result and exact-score accuracy, but a clearer gain in Scoreline. New schedules can be added as they begin, allowing the benchmark to evaluate future models without using outcomes that are already known. Code, predictions and evaluation scripts will be publicly released.

足球预测大模型评测动态基准

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。