用大模型从用户行为序列生成高质量嵌入，提升金融、医疗等场景的用户分类效果。
LLM4ES: Learning User Embeddings from Event Sequences via Large Language Models
- 将事件序列转为文本，用大模型自监督训练生成嵌入
- 在金融等低多样性领域表现优于现有方法
- 适合需要精准用户表征的应用，如金融风控、医疗预测
本文提出 LLM4ES，一种利用预训练大语言模型从事件序列中学习用户嵌入的新框架。将事件序列转换为文本表示，通过下一步词预测任务微调大模型，生成高质量用户嵌入。引入文本增强技术，提升大模型对事件序列数据的适应性，显著改善低多样性领域的表示质量。实验表明，LLM4ES 在金融等领域的用户分类任务中达到当前最优性能，优于已有嵌入方法。生成的用户嵌入可广泛应用于用户分群、患者预后预测等场景。
原文摘要 · Abstract (English)
This paper presents LLM4ES, a novel framework that exploits large pre-trained language models (LLMs) to derive user embeddings from event sequences. Event sequences are transformed into a textual representation, which is subsequently used to fine-tune an LLM through next-token prediction to generate high-quality embeddings. We introduce a text enrichment technique that enhances LLM adaptation to event sequence data, improving representation quality for low-variability domains. Experimental results demonstrate that LLM4ES achieves state-of-the-art performance in user classification tasks in financial and other domains, outperforming existing embedding methods. The resulting user embeddings can be incorporated into a wide range of applications, from user segmentation in finance to patient outcome prediction in healthcare.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。