用自监督学习构建银行事件序列的通用模型，直接从原始数据预测信用与风险。
PRAGMA: Revolut Foundation Model
- 基于Transformer的自监督预训练，处理可变长度金融事件序列。
- 仅用线性分类器在嵌入上训练即达强性能，微调后效果更优。
- 适合金融风控、信用评估等场景，无需复杂特征工程。
现代金融系统生成海量交易与事件级数据，蕴含丰富的经济信号。本文提出PRAGMA，一种面向银行事件序列的基础模型家族。通过在大规模异构银行事件语料上使用针对离散、可变长度金融记录设计的自监督目标，对基于Transformer的架构进行预训练。所得模型支持多种下游任务，如信用评分、欺诈检测和客户生命周期价值预测：仅需在提取的嵌入上训练简单线性模型即可获得优异性能，轻量微调可进一步提升效果。在多个下游任务上的广泛评估表明，PRAGMA能直接从原始事件序列中实现跨领域的卓越表现，为金融应用提供通用表征层。
原文摘要 · Abstract (English)
Modern financial systems generate vast quantities of transactional and event-level data that encode rich economic signals. This paper presents PRAGMA, a family of foundation models for banking event sequences. Our approach pre-trains a Transformer-based architecture with masked modelling on a large-scale, heterogeneous banking event corpus using a self-supervised objective tailored to the discrete, variable-length nature of financial records. The resulting model supports a wide range of downstream tasks such as credit scoring, fraud detection, and lifetime value prediction: strong performance can be achieved by training a simple linear model on top of the extracted embeddings and can be further improved with lightweight fine-tuning. Through extensive evaluation on downstream tasks, we demonstrate that PRAGMA achieves superior performance across multiple domains directly from raw event sequences, providing a general-purpose representation layer for financial applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。