用连续向量输入建模外汇收益,提升金融序列预测精度。
VAIOM: Continuous-Input, Discrete-Output Decoder-Only Financial Sequence Modeling
- 输入用连续金融事件向量,输出用离散收益桶分类。
- 相比基准模型,每事件提升0.029~0.043比特的预测性能。
- 适合关注高精度金融时序建模的研究者与量化从业者。
金融数据是连续、异构且噪声多的,而传统解码器仅模型通常基于离散符号输入。本文提出VAIOM——一种用于一小时外汇行情棒的连续输入、离散输出的解码器式Transformer,实现概率化下一收益建模。该模型将输入表示与输出似然分离:输入保留多变量金融事件的数值结构,输出采用归一化波动率的收益桶分类分布,支持交叉熵训练与似然评估。所选0.9M参数混合连续输入模型结合了连续事件特征与离散资产元数据,采用市场状态混合的收益头,辅以间隙、波动率制度与序数辅助目标,并使用全序列监督。模型与预处理基于2024年前训练数据拟合;在2024下半年验证集上选择模型,不重新拟合,在两个2025测试期进行评估。三个独立训练种子下,所有模型均优于固定单棒LightGBM基线,最优检查点在两个测试期分别获得0.029与0.043比特/事件的提升。验证实验表明:连续输入优于离散输入;全序列监督优于仅最后位置训练;辅助表征塑造与混合结构收益头共同提升收益似然。容量研究发现,最小完整架构在当前语料库上取得最强验证似然。
原文摘要 · Abstract (English)
Financial observations are continuous, heterogeneous, and noisy, whereas decoder-only next-token models are usually built around discrete symbolic inputs. We introduce Vector-Input Autoregressive Inference for Ordinal-Return Modeling (VAIOM), a decoder-only Transformer for probabilistic next-return modeling on one-hour foreign-exchange bars. VAIOM separates input representation from output likelihood: continuous multivariate financial-event vectors preserve numerical structure at the input, while a categorical distribution over the next volatility-normalized return bucket supports cross-entropy training and likelihood evaluation. The selected 0.9M Hybrid Continuous Input model combines continuous event features with categorical asset metadata, a Mixture-of-Market-States return head, Gap, volatility-regime, and Ordinal auxiliary objectives, and full-sequence supervision. Models and preprocessing are fit using pre-2024 Train data; models are selected on 2024H2 Validation and evaluated without refitting on two 2025 Test periods. Across three independent training seeds, every model outperforms fixed single-bar LightGBM baseline in both Test halves. For the canonical checkpoint, paired gains over LightGBM are 0.029 and 0.043 bits per event. Validation experiments show that continuous input improves over discrete-token input under the same categorical return objective, full-sequence supervision improves over last-position training, and auxiliary representation shaping together with a mixture-structured return head improves return likelihood in controlled comparisons. A supporting capacity study finds that the smallest evaluated complete architecture rung achieves the strongest Validation likelihood on the present corpus.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。