将文本与时间序列结合,用字节令牌提升网页事件建模效果
Byte-token Enhanced Language Models for Temporal Point Processes Analysis
- 用字节令牌编码时间间隔,直接接入大语言模型架构
- 在电商、社交等数据集上实现顶尖的事件时间与类型预测性能
- 可生成更真实描述和情绪更匹配的事件内容,适合内容生成研究
时间点过程(TPPs)广泛用于建模网络中的事件序列,如用户评论、社交媒体帖子和在线交易。然而传统TPP模型难以有效利用伴随事件的丰富文本描述,而大语言模型虽具备强大文本处理能力,却缺乏对网络事件序列中固有时序动态的建模机制。为此,我们提出Language-TPP,一个统一框架,将TPPs与大语言模型无缝融合,以增强网络事件序列建模。其核心创新是一种新型时序编码机制,将连续时间间隔转化为专用字节令牌,使标准语言模型架构可直接用于TPP建模,无需特定修改。该方法在多个真实世界网络数据集(涵盖电商评论、社交媒体和在线问答平台)上的事件时间预测与类型预测任务中均达到当前最优表现。更重要的是,该统一框架拓展了TPP研究新能力:引入时序信息后,生成事件描述的质量显著提升(ROUGE-L得分提高),情感分布也更合理。通过全面实验,包括对学习分布的定性分析及长序列可扩展性评估,证明Language-TPP能有效捕捉网络用户行为中的时序动态与文本模式,对内容生成、用户行为理解及平台应用具有重要意义。代码已开源。
原文摘要 · Abstract (English)
Temporal Point Processes (TPPs) have been widely used for modeling event sequences on the Web, such as user reviews, social media posts, and online transactions. However, traditional TPP models often struggle to effectively incorporate the rich textual descriptions that accompany these events, while Large Language Models (LLMs), despite their remarkable text processing capabilities, lack mechanisms for handling the temporal dynamics inherent in Web-based event sequences. To bridge this gap, we introduce Language-TPP, a unified framework that seamlessly integrates TPPs with LLMs for enhanced Web event sequence modeling. Our key innovation is a novel temporal encoding mechanism that converts continuous time intervals into specialized byte-tokens, enabling direct integration with standard language model architectures for TPP modeling without requiring TPP-specific modifications. This approach allows Language-TPP to achieve state-of-the-art performance across multiple TPP benchmarks, including event time prediction and type prediction, on real-world Web datasets spanning e-commerce reviews, social media and online Q&A platforms. More importantly, we demonstrate that our unified framework unlocks new capabilities for TPP research: incorporating temporal information improves the quality of generated event descriptions, as evidenced by enhanced ROUGE-L scores, and better aligned sentiment distributions. Through comprehensive experiments, including qualitative analysis of learned distributions and scalability evaluations on long sequences, we show that Language-TPP effectively captures both temporal dynamics and textual patterns in Web user behavior, with important implications for content generation, user behavior understanding, and Web platform applications. Code is available at https://github.com/qykong/Language-TPP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。