TS-Arena通过预注册机制实时评估时间序列模型,杜绝数据泄露。
TS-Arena -- A Live Forecast Pre-Registration Platform
- 模型需在真实数据出现前提交预测,避免测试污染。
- 一年能源数据实测显示,主流模型长期表现稳定,新模型可快速验证。
- 适合追求真实泛化能力的时序预测研究者和工业应用团队。
时间序列基础模型(TSFMs)正在重塑预测领域。然而,基于历史数据评估时,训练-测试样本重叠与时间序列相关性带来的时序重叠风险日益加剧。为此,我们提出TS-Arena——一个将评估从已知过去转向未知未来的实时预测平台。基于持续基准测试理念,TS-Arena在真实未来数据上评估模型。关键在于引入严格的预测预注册协议:模型必须在真实数据存在前提交预测,从设计上杜绝测试集污染。平台采用模块化微服务架构,整合多源数据并协调容器化模型提交。通过在实时数据流上强制预注册,TS-Arena有效防止信息泄露,提供比传统静态、低频竞赛(如M-Competitions)更快的评估方式。一年能源时间序列的初步实证结果表明,现有TSFMs表现出稳健的长期得分,同时平台连续性使新模型能即时展现竞争力。TS-Arena为评估现代预测模型的真实泛化能力提供了必要基础设施。平台与代码已公开于https://ts-arena.live/。
原文摘要 · Abstract (English)
Time Series Foundation Models (TSFMs) are transforming the field of forecasting. However, evaluating them on historical data is increasingly difficult due to the risks of train-test sample overlaps and temporal overlaps between correlated train and test time series. To address this, we introduce TS-Arena, a live forecasting platform that shifts evaluation from the known past to the unknown future. Building on the concept of continuous benchmarking, TS-Arena evaluates models on future data. Crucially, we introduce a strict forecasting pre-registration protocol: models must submit predictions before the ground-truth data physically exists. This makes test-set contamination impossible by design. The platform relies on a modular microservice architecture that harmonizes and structures data from different sources and orchestrates containerized model submissions. By enforcing a strict pre-registration protocol on live data streams, TS-Arena prevents information leakage offers a faster alternative to traditional static, infrequently repeated competitions (e.g. the M-Competitions). First empirical results derived from operating TS-Arena over one year of energy time series demonstrate that established TSFMs accumulate robust longitudinal scores over time, while the continuous nature of the benchmark simultaneously allows newcomers to demonstrate immediate competitiveness. TS-Arena provides the necessary infrastructure to assess the true generalization capabilities of modern forecasting models. The platform and corresponding code are available at https://ts-arena.live/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。