用指数预训练提升股票收益预测，效果优于传统模型。
From Index to Equity: Pre-Training Transformers for Stock Return Prediction
- 先在指数上预训练，再微调到个股，提升预测能力。
- 预训练使股票预测的交叉熵损失从0.69降至0.64。
- 实现实时预测系统，适合量化交易与投资决策参考。
本研究旨在利用机器学习提升股票价格预测能力，辅助买卖持有等投资决策。具体而言，探索基于Transformer的股票预测模型，并考察预训练策略对预测性能的影响。首先在多伦多证券交易所指数（TSX）上预训练变压器模型，以预测日内收益方向，随后在个别TSX股票上进行微调，用于收益值回归任务。性能与长短期记忆网络（LSTM）和XGBoost模型进行对比。在个股预测中，指数预训练将二元交叉熵损失从0.69降低至0.64。微调后的变压器回归模型均方误差低于基准模型，尽管集成模型和XGBoost模型实现了更高的平均每日收益率。此外，开发了实时股票预测应用系统，支持交易决策。未来工作将聚焦于扩大模型容量、引入更广泛的全球技术指标，并剔除可预测性低的股票。
原文摘要 · Abstract (English)
This research aims to leverage machine learning to improve stock price prediction and support informed investment decisions related to buying, selling, and holding assets. Specifically, this work investigates transformer-based models for stock prediction and examines the impact of pre-training strategies on forecasting performance. A transformer model was first pre-trained on the Toronto Stock Exchange Index (TSX) to predict intra-day return direction and subsequently fine-tuned on individual TSX stocks. The model was further adapted for return-value regression tasks. Performance was benchmarked against Long Short-Term Memory (LSTM) and XGBoost models. Pre-training on the market index improved the binary cross-entropy loss for individual stock prediction from 0.69 to 0.64. The fine-tuned transformer regression model achieved lower mean squared error than the benchmark models, although the ensemble and XGBoost models achieved higher average daily returns. In addition, a practical application was developed to deliver real-time stock predictions for trading support. Future work will focus on increasing transformer model capacity, incorporating broader global technical indicators, and filtering out stocks with low predictability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。