arXiv:2608.03063cs.CL2026-08

将行为序列建模能力注入大模型,提升支付平台风控精准率。

SeqLLM: Augmenting LLMs with Behavioral-Sequence Modeling for High-Stakes Decisions at WeChat Pay

论文配图:SeqLLM: Augmenting LLMs with Behavioral-Sequence Modeling for High-Stakes Decisions at WeChat Pay
图 1 · 摘自论文原文
  • 用离散行为词表和轻量投影器,让大模型理解用户行为序列。
  • 精度从92.0%提升至97.5%,在百万级商户中降低误判。
  • 适合需要融合文本与长序列的高风险决策场景。

大型支付平台每日需筛查数千万商家,误报损害合规商户,漏报则留下风险。最棘手的情况需同时理解商家的文本资料与长期行为序列。大语言模型擅长文本处理,但难以原生建模序列,且改造常导致灾难性遗忘。本文提出SeqLLM框架,在保留语言能力的前提下,为预训练大模型注入行为序列建模能力。其包含三个组件:紧凑的离散行为词表,将行为事件映射为原生令牌;轻量级投影器,通过两阶段对齐课程训练,将这些令牌嵌入大模型语义空间;以及前缀引导的能力注入机制,通过任务前缀监督微调获取序列建模能力,而非持续预训练。该模型已部署于微信支付,每日筛查数百万商家。相比生产环境基于DeepSeek的大模型基线,筛查精确率由92.0%提升至97.5%。其预训练的行为令牌嵌入还使一个服务于百亿级交易流量的欺诈检测器在Top-0.01%的召回精度上提升26.8个百分点。超越支付领域,SeqLLM在公开推荐基准上也取得领先:在MovieLens和Amazon上,相对强基线User-LLM的Recall@5提升最高达32%;在RecIF上,以仅五分之一的GPU日耗,超越完整OneRec-8B流水线14.2%的Pass@32。

原文摘要 · Abstract (English)

Merchant risk control at large payment platforms screens tens of millions of merchants daily, where false positives harm legitimate merchants and false negatives leave harmful activity undetected. The hardest cases require jointly understanding a merchant's textual profile and long behavioral sequence. Large language models (LLMs) excel at text but cannot natively model such sequences, while adapting them often causes catastrophic forgetting. We present SeqLLM, a framework that adds behavioral-sequence modeling to a pretrained LLM while preserving its language ability. SeqLLM combines three components: a compact discrete vocabulary that represents behavioral events as native tokens; a lightweight projector, trained with a two-stage alignment curriculum, that grounds these tokens in the LLM's semantic space; and prefix-guided capability injection, which acquires sequence-modeling ability through task-prefixed supervised fine-tuning rather than continual pre-training. SeqLLM is deployed at WeChat Pay, screening millions of merchants daily. Against the production DeepSeek-based LLM baseline, it raises screening precision from 92.0% to 97.5%. Its pretrained behavior-token embeddings also improve [email protected]% by 26.8 percentage points in a production fraud detector serving billion-scale transaction traffic. Beyond payments, SeqLLM achieves state-of-the-art results on public recommendation benchmarks. On MovieLens and Amazon, it surpasses the strong User-LLM baseline by up to 32% relative Recall@5 while retaining markedly stronger language ability. On RecIF, it improves Pass@32 by 14.2% over the full OneRec-8B pipeline using only one-fifth of its GPU-days.

大模型行为序列风控推荐系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。