通过延迟思维链提升时间序列预测精度，模型输出越长效果越好。
Output Scaling: YingLong-Delayed Chain of Thought in a Large Pretrained Time Series Forecasting Model
- 采用非因果双向注意力结构，通过延迟思维链推理增强预测能力。
- 在多个数据集上实现超过60%的零样本性能，300M模型领先44%。
- 适合需要高精度时间序列预测的研究者与工业应用者。
我们提出一种联合预测框架，突破传统直接或递归方法的局限。该框架基于自研基础模型YingLong，其为非因果、双向注意力编码器-仅变压器，通过掩码标记恢复训练，更适配语言理解任务。实验发现，输出长度越长，模型精度越高，源于延迟思维链推理机制。通过多输入集成有效降低输出方差。释放了参数量从6M到300M的四款模型，在ETT和Weather数据集上零样本表现优异，最高达60%以上。使用23个跨7个领域的GIFT-Eval基准评估，300M模型相较最优时序基础模型和端到端模型分别提升14%和44%排名。预训练300M模型已公开于HuggingFace。
原文摘要 · Abstract (English)
We present a joint forecasting framework for time series prediction that contrasts with traditional direct or recursive methods. This framework achieves state-of-the-art performance for our designed foundation model, YingLong, and reveals a novel scaling effect: longer outputs significantly enhance model accuracy due to delayed chain-of-thought reasoning in our non-causal approach. YingLong is a non-causal, bidirectional attention encoder-only transformer trained through masked token recovery, aligning more effectively with language understanding tasks than with generation tasks. Additionally, we boost performance by tackling output variance with a multi-input ensemble. We release four foundation models ranging from 6M to 300M parameters, demonstrating superior results in zero-shot tasks on the ETT and Weather datasets. YingLong achieves more than 60% best performance. To ensure generalizability, we assessed the models using the GIFT-Eval benchmark, which comprises 23 time series datasets across 7 domains. Yinglong significantly outperformed the best time-series foundation models, end-to-end trained models by 14% and 44% in rank respectively.The pretrained 300M model is available at https://huggingface.co/qcw1314/YingLong_300m
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。