探索大模型处理事件序列的时间标记策略，发现匹配数据分布才能提升性能。
Temporal Tokenization Strategies for Event Sequence Modeling with Large Language Models
- 对比五种时间编码方式，包括数值字符串、字节级、日历标签等。
- 不同数据分布下表现差异大，无通用最优策略。
- 提醒研究者关注时间标记与数据特性的匹配性，适合时序建模方向。
用大语言模型（LLM）建模时间事件序列时，连续时间的表示是一个关键且研究不足的问题。已有方法如字节级表示或日历标记存在局限，尤其在面对真实世界事件数据的多样化分布时——从平滑的对数正态分布到离散突变模式。本文系统评估了五种时间标记策略：朴素数值字符串、高精度字节级表示、人类语义日历标记、经典均匀分箱和自适应残差标量量化。在体现这些分布特性的实际数据集上微调LLM进行测试。结果表明，没有单一策略在所有场景下最优；预测性能高度依赖于标记器与数据统计特性的一致性，凸显时间标记作为LLM事件建模中常被忽视的关键设计维度。
原文摘要 · Abstract (English)
Representing continuous time is a critical and under-explored challenge in modeling temporal event sequences with large language models (LLMs). Various strategies like byte-level representations or calendar tokens have been proposed. However, the optimal approach remains unclear, especially given the diverse statistical distributions of real-world event data, which range from smooth log-normal to discrete, spiky patterns. This paper presents a systematic empirical study of temporal tokenization for modeling event sequences with LLMs, comparing distinct encoding strategies: naive numeric strings, high-precision byte-level representations, human-semantic calendar tokens, classic uniform binning, and adaptive residual scalar quantization. We evaluate these strategies by fine-tuning LLMs on real-world datasets that exemplify these diverse distributions. Our analysis reveals that no single strategy is universally superior; instead, prediction performance depends heavily on aligning the tokenizer with the data's statistical properties, highlighting temporal tokenization as a critical yet often overlooked design dimension in LLM-based event modeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。