arXiv:2512.08957cs.LGcs.AI2025-12

LUMOS用大模型统一预测用户行为,无需人工特征工程。

LUMOS: Large User MOdels for User Behavior Prediction

  • 用Transformer联合学习原始用户数据,免去任务专用模型和特征工程。
  • 在5个任务上平均提升0.025的ROC-AUC,回归任务MAPE降低4.6%。
  • 适合需要大规模用户行为预测的平台,尤其关注业务效果落地。

大规模用户行为预测仍是B2C平台的核心挑战。传统方法依赖任务专用模型和领域特定特征工程,耗时耗力且需专业知识,难以扩展。我们提出LUMOS(Large User MOdel Series),一种基于Transformer的架构,通过仅使用原始用户活动数据联合学习多个任务,消除任务专用模型和人工特征工程。LUMOS引入新型跨注意力机制,利用未来已知事件(如节假日、促销等)进行条件预测,可捕捉“即将到来的节假日如何影响用户参与度”等复杂模式。该架构采用多模态标记化,将用户行为、事件上下文与静态用户人口统计属性融合为丰富表征,经专用嵌入路径处理。在包含17亿用户活动记录(覆盖2.5亿用户)的生产数据集上,实验表明LUMOS性能显著优于传统方法:在5个任务中,二分类任务平均ROC-AUC提升0.025,回归任务MAPE降低4.6%。线上A/B测试验证,模型优化带来3.15%的日活用户增长,具备显著商业价值。

原文摘要 · Abstract (English)

User behavior prediction at scale remains a critical challenge for online B2C platforms. Traditional approaches rely heavily on task-specific models and domain-specific feature engineering. This is time-consuming, computationally expensive, and requires domain expertise and therefore, not scalable. We present LUMOS (Large User MOdel Series), a transformer-based architecture that eliminates task-specific models and manual feature engineering by learning multiple tasks jointly using only raw user activity data. LUMOS introduces a novel cross-attention mechanism that conditions predictions on future known events (e.g., holidays, sales, etc.), enabling the model to predict complex behavior patterns like "how will upcoming holidays affect user engagement?" The architecture also employs multi-modal tokenization, combining user activities, event context, and static user demographic attributes into rich representations processed through specialized embedding pathways. Through extensive experiments on a production dataset spanning 1.7 trillion user activity tokens from 250 million users, we demonstrate that LUMOS achieves superior performance compared to traditional task-specific models. Across 5 tasks with established baselines, we achieve an average improvement of 0.025 in ROC-AUC for binary classification tasks and 4.6\% reduction in MAPE for regression tasks. Online A/B testing validates these improvements translate to measurable business impact with a 3.15\% increase in Daily Active Users.

用户行为预测大模型多任务学习A/B测试

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。