用统一事件序列预训练大模型，提升金融多任务预测效果。
A Foundation Model for Multimodal Event Sequences in Financial Applications
- 将多源事件统一为时间序列，通过自回归预测学习通用表示。
- 在银行生产环境部署后，多个业务指标显著提升。
- 适合需要融合异构数据的金融场景，降低模型开发成本。
预测建模是现代金融服务的核心，传统方法依赖人工设计的表格特征，针对不同任务分别建模，限制了模型复用，难以充分利用交易记录和数字交互等异构数据。本文提出一种基于基础变换器模型的预训练方法，将来自多源的数据事件统一为单一时间序列，实现异构模态的早期融合，并通过下一项事件预测目标学习通用表示。这些表示与现有工程化用户特征结合，在此基础上训练轻量级神经网络完成多项下游任务。该系统在东欧最大银行之一的生产环境中部署，显著优于传统任务专用模型，同时降低开发成本，带来可衡量的业务指标改善。
原文摘要 · Abstract (English)
Predictive modeling is a core component of modern financial services, where a wide range of tasks are traditionally addressed using separate models trained on manually engineered tabular features. This task-specific approach limits reuse and makes it difficult to fully exploit heterogeneous data sources such as transaction histories and digital interaction signals. In this paper, we present an approach based on pretraining a foundation transformer model on multimodal sequences of user events. Events from multiple data sources are unified into a single chronological sequence, enabling early fusion of heterogeneous modalities and learning of general-purpose representations via a next-event prediction objective. These representations are combined with existing engineered user features, on top of which lightweight neural models are trained for multiple downstream tasks. The proposed system outperforms traditional task-specific models while reducing development overhead. The approach was deployed in production at one of the biggest banks in Eastern Europe, resulting in measurable improvements in business metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。