用大模型嵌入提升交易理解，兼顾效果与效率
Enhancing Foundation Models in Transaction Understanding with LLM-based Sentence Embeddings
- 用大模型生成语义嵌入，替代传统离散编码
- 在多个任务上显著提升交易理解性能
- 适合需要高效又精准的金融场景应用
支付网络的普及产生了海量交易数据，蕴含丰富的消费者与商户行为模式。当前交易分析的基础模型虽能处理表格数据，但对商户类别字段仍依赖基于索引的表示方式，将丰富文本信息转化为离散标记，造成大量语义损失。尽管大语言模型（LLM）具备更强的语义理解能力，其计算开销又难以满足实时金融部署需求。本文提出一种混合框架，利用LLM生成的嵌入作为轻量级交易模型的语义初始化，在可解释性与运行效率间取得平衡。方法融合多源数据以丰富商户类别字段，并采用单字约束原则确保不同LLM架构下嵌入的一致性。通过噪声过滤与上下文感知增强系统性提升数据质量。在大规模交易数据集上的实验表明，该方法在多个交易理解任务中均实现显著性能提升。
原文摘要 · Abstract (English)
The ubiquity of payment networks generates vast transactional data encoding rich consumer and merchant behavioral patterns. Recent foundation models for transaction analysis process tabular data sequentially but rely on index-based representations for categorical merchant fields, causing substantial semantic information loss by converting rich textual data into discrete tokens. While Large Language Models (LLMs) can address this limitation through superior semantic understanding, their computational overhead challenges real-time financial deployment. We introduce a hybrid framework that uses LLM-generated embeddings as semantic initializations for lightweight transaction models, balancing interpretability with operational efficiency. Our approach employs multi-source data fusion to enrich merchant categorical fields and a one-word constraint principle for consistent embedding generation across LLM architectures. We systematically address data quality through noise filtering and context-aware enrichment. Experiments on large-scale transaction datasets demonstrate significant performance improvements across multiple transaction understanding tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。